An introduction to the AI Technology Stack

Artificial Intelligence (AI) is integrated into our core business processes and daily workflows. You’ve heard the buzz, but how does it work? It’s actually a combination of different technologies. That’s what we’ll break down in this post.

I have divided AI into functional layers and created a tower diagram to guide our discussion.

The Foundation: Hardware and Chips

Computer hardware and chips are the foundation layer for AI. They are the components that perform the programmed calculations. We’re talking about powerful CPU-based servers and workstations integrated with specialized processors like GPUs.

AI is not only about algorithms and models, but also about the underlying physical infrastructure that makes higher-level abstractions possible.

The Fundamentals: Data

AI systems fundamentally require data for input. Imagine data as patterns that energize fluctuations in a complex network. Each data point—whether a customer record, a sales figure, a product image, or text—contributes to the overall flow and output of the system.

This diverse data landscape is used by AI to evaluate and generate results. The quality and quantity of data directly impact AI performance.

The Network Infrastructure: Cloud

How is AI data exchanged and saved? The cloud, or network, layer is the infrastructure that interacts with and backs up the devices you use. AI processes run on a shared cloud of computers that automatically scale up or down, and restart, as needed.

Commercial cloud is not the only option. You can also maintain a private network and keep everything on-premises, meaning that all the hardware and software is located at your place of business.

The Tools: Frameworks and Toolkits

AI developers don’t build from scratch; they use frameworks and toolkits. These are software resources that serve as building blocks for creating AI systems. They allow engineers to design the flow of data processing.

This level provides access to foundation models, and supports the creation of powerful AI applications.

Pre-Built Components: Foundation Models

Only a small group of large high-tech companies create what are known as foundation models. These are pre-trained AI models that have already learned from massive amounts of data, providing the base for a wide range of applications. They avoid the need to train models from scratch and accelerate innovation.

This level is a powerful shortcut, that can be adapted for various tasks, such as generating text or images, and recognizing patterns that are otherwise undetectable.

The Final Product: AI Applications

At the top of the AI tech stack are user-facing AI applications. These are designed to be intuitive, easy to use, and keep humans in the loop (HITL). Examples include a chatbot that answers your questions, a recommendation system that suggests your next query, or an image app that creates the pictures you describe.

The application layer we interact with every day is where AI becomes genuinely useful. It is the culmination of all the layers below, working together to put the power of AI beneath our fingertips.

Putting it all together

With the layers of the AI Tech Stack explored, we can see how it all comes together. For me, working with this framework helps make sense of the fast-paced and sometimes bewildering changes in the tech world. I hope this framework helps you frame your own thoughts about the evolution of artificial intelligence.

Chain of thought geometric reasoning

The technique of Chain-of-Thought (CoT) prompting evaluates the ability of an AI model to break complex problems into a series of logical steps. To validate the integrity and reproducibility of the expected results, two volunteers successfully completed the exercise embedded in this paper, prior to querying several large language models (LLMs).

The LLMs were prompted to evaluate a straightedge and compass construction illustrating Definitions 3 & 4 from Euclid’s “Elements”, Book III. Two open-source models ran in a laboratory environment on a single GPU isolated from the Internet. The remaining models were tested online.

No, none of the straight lines 1, 2, 3, or 4 are at a greater distance from the center of the circle. Due to the symmetric nature of the construction.

— Claude 3.5 (The construction is actually rectangular)

Many of the LLMs correctly parsed the logic but were unable to reach a conclusion absent exact measurement. Note that Euclid’s approach in “Elements” was to emphasize general principles and logical reasoning rather than numerical calculation.

Some of the LLMs gave contradictory answers within the same response. One model produced an SVG diagram of a squared-off construction, whereas the instructions specifically stated rectangular.

The steps are easy to complete. The answer to both questions is yes. None of the LLMs successfully answered the prompt.

Understanding Euclidean Geometric Construction

Euclid is one of the most influential figures in the history of mathematics. He is known for a comprehensive description of geometry, “Elements”. Its proofs are known for their logical rigor and their systematic approach. In this essay I illustrate simple straightedge and compass construction.

Clarifying definitions by illustration

The essay in Part 1 of this series required both mathematical understanding and a high level of reading comprehension. This helped us to evaluate Artificial Intelligence (AI) Large Language Models (LLMs) on both of those challenges.

In this part we want to make geometric concepts as easy to understand as possible.

“Elements” Book III begins with a number of Definitions. However, reading through them we find there’s a lot to unpack here. Let’s try to make these easier to understand with a drawing (Figure1).

The definitions

4. In a circle straight lines are said to be equally distant from the center when the perpendiculars drawn to them from the center are equal.

5. And that straight line is said to be at a greater distance on which the greater perpendicular falls.

— Elements, Book 3 Definitions

Geometry exercise

Tools:

• Blank sheet of paper
• Straightedge
• Compass
• Red marker
• Orange marker

Using a straightedge and compass complete the following steps

1. On a blank sheet of rectangular paper draw a line from the top-left corner to the bottom right corner of the page.
2. Draw a line from the bottom left corner to the top right corner.
The lines create an intersection at the center of the page.
3. Set the compass to a width that allows a complete circle small enough to fit on the page.
From the center of the page draw a circle.
4. Label the point at the top-left where the line crosses the circle, point A.
   Label the point at the top-right where the line crosses the circle, point B.
   Label the point at the bottom-right where the line crosses the circle, point C.
   Label the point at the bottom-left where the line crosses the circle, point D.
5. Without changing the compass, use it to mark two points on the circle, starting from point B.
   Label the new point between A and B as point E.
   Label the new point between B and C as point F.
6. Without changing the compass, use it to mark two points on the circle starting from point D.
   Label the new point between C and D as point G.
   Label the new point between D and A as point H.
7. Draw a straight line from point E to point F and label it line 1.
   Draw a straight line from point G to point H and label it line 2.
   Draw a straight line from point B to point G and label it line 3.
   Draw a straight line from point E to point H and label it line 4.
8. Using the compass, bisect lines 1, 2, 3 and 4 to find the center of each line.
9. From the center point of lines 1, 2, 3, and 4 draw perpendicular lines to the center point of the circle.
10. For this exercise make the perpendiculars from lines 1, 2, and 3 red.
    Make the perpendicular from line 4 orange.

Answer the following questions

1. Are any of the straight lines 1, 2, 3, or 4 at a greater distance from the center of the circle?
2. Is the orange line longer than any of the red?

Appendix A: Drawings by volunteers

Appendix B: CoT prompt 2

Appendix C: Anthropic Claude 3

“No, the orange line (perpendicular to line 4) is not longer than any of the red lines.”

The construction is squared-off, producing errors.

Appendix D: X Grok

“The perpendiculars from the center to these chords would be equal if the chords were at equal distances from the center, which they are not in this setup.” Ergo, the answer is yes and not “we can’t definitively say.”

Appendix E: OpenAI ChatGPT

“The answer here would depend on the relative lengths of these perpendiculars, which could vary depending on the exact configuration of points on your paper.” Dodges the question.

Appendix F: Google Gemini

“No. All the lines 1, 2, 3, and 4 are equidistant from the center of the circle.” Contradicts the answer to question 2, “Yes. The orange line (line 4) is longer than any of the red lines… The orange line is a diameter of the circle.”

Appendix G: Meta Llama 3.2 (open-source. lab test)

“No. based on the instructions, the orange line is not longer than any of the red lines..”

Appendix H: Microsoft Phi-3 (open source. lab test)

“The orange line (which is the perpendicular from line 4) is not necessarily longer than any of the red lines.”

Appendix I: Summary of LLM release notes and benchmarks

Anthropic Claude 3

Opus, our most intelligent model, outperforms its peers on most of the common evaluation benchmarks for AI systems, including undergraduate level expert knowledge (MMLU), graduate level expert reasoning (GPQA), basic mathematics (GSM8K), and more. It exhibits near-human levels of comprehension and fluency on complex tasks, leading the frontier of general intelligence.

GSM8K is a dataset of linguistically diverse grade school math word problems requiring multi-step reasoning.

X Grok

Both Grok-2 and Grok-2 mini demonstrate significant improvements over our previous Grok-1.5 model. They achieve performance levels competitive to other frontier models in areas such as graduate-level science knowledge (GPQA), general knowledge (MMLU, MMLU-Pro), and math competition problems (MATH). Additionally, Grok-2 excels in vision-based tasks, delivering state-of-the-art performance in visual math reasoning (MathVista) and in document-based question answering (DocVQA).

MathVista is a benchmark designed to evaluate the mathematical reasoning capabilities of AI models, particularly in visual contexts.

OpenAI ChatGPT

We’ve created GPT-4, the latest milestone in OpenAI’s effort in scaling up deep learning. GPT-4 is a large multimodal model (accepting image and text inputs, emitting text outputs) that, while less capable than humans in many real-world scenarios, exhibits human-level performance on various professional and academic benchmarks.

SAT (Scholastic Assessment Test) is a standardized test widely used for college admissions in the United States. The Math section assesses mathematical skills and problem-solving abilities, including geometry.

Google Gemini

We designed Gemini to be natively multimodal, pre-trained from the start on different modalities. Then we fine-tuned it with additional multimodal data to further refine its effectiveness. This helps Gemini seamlessly understand and reason about all kinds of inputs from the ground up.

The MATH benchmark focuses on a broad range of mathematical concepts, including algebra, geometry, and calculus.

Meta Llama 3

In the development of Llama 3, we looked at model performance on standard benchmarks and also sought to optimize for performance for real-world scenarios. To this end, we developed a new high-quality human evaluation set. This evaluation set contains 1,800 prompts that cover 12 key use cases: asking for advice, brainstorming, classification, closed question answering, coding, creative writing, extraction, inhabiting a character/persona, open question answering, reasoning, rewriting, and summarization.

The ARC Challenge (Abstraction and Reasoning Corpus) is a benchmark designed to evaluate an AI model’s ability to solve problems requiring abstract reasoning and pattern recognition.

Microsoft Phi-3

Phi-3 models significantly outperform language models of the same and larger sizes on key benchmarks.

The OpenBookQA benchmark is a dataset designed to evaluate an AI model’s understanding of core science facts and their application to novel situations.

Chain of thought prompting

The technique of Chain-of-Thought (CoT) prompting evaluates the ability of an AI model to break complex problems into a series of logical steps.  The essay embedded in this post was applied to generative artificial intelligence (GenAI) models to test their ability to process geometric numerical analysis. Their outputs offer insight into breakthroughs at the forefront AI chatbot development.

Critical Thinking

The square of the hypotenuse (156.25) is actually greater than the sum of the squares on the sides (241.25) by 85.

— Meta Llama 3.1 8B-Instruct on scaled down hardware. (156.25 is actually less than 241.25)

CoT prompting was applied to two open-source large language models (LLMs) running on a single GPU isolated from the Internet. Under these conditions smaller models produced output that is clearly wrong when examined with critical thinking.

One model asserted that the essay was not entirely correct because 156.25 is greater than 241.25. A second model failed to process geometric logic and failed at number substitution.

The prompt was also applied to large-scale GenAI models running on cloud servers. These evaluated the essay correctly. The outputs of four models are shown as screenshots in the appendices to this post.

Not tested was the recently announced OpenAI o1. This new series of models was designed to spend more time processing before they respond, in order to solve harder problems in science, coding and math. Such as the essay below.

(As an aside to my regular readers, the AiJunzi platform was upgraded to Microsoft Phi 3.5 since my last post.)

Discerning Pythagorean and Euclidean Logic

Pythagoras and Euclid are two of the most influential figures in the history of mathematics.  Pythagoras is best known for the Pythagorean Theorem, which states that the square of the hypotenuse on a right triangle is equal to the sum of the squares of the other two sides.

a² + b² = c²

Euclid wrote a comprehensive description of geometry. The proofs in Euclid’s “Elements” are known for their logical rigor and their systematic approach.  In this essay I propose that the Euclidean method can be explained using numerical analysis.

Numerical analysis concisely verifies mathematical propositions

One of Euclid’s proofs is a statement of the Pythagorean Theorem.  However, reading through Proposition 47, we find that it relies on earlier propositions, which themselves rely on earlier postulates.  This can make it difficult to understand the underlying logical principles.  There’s a lot to unpack here.  We can use the Pythagorean Theorem and numeric values to provide an intuitive understanding of one of Euclid’s propositions.

Acute triangles and squared sides by the numbers

Let’s look at Book 2, Proposition 13 using numerical analysis. Referring to Figure 1 (top) and part of the proof (below) we can simply state that (AB)² = (BD)² + (DA)².

But the square on AB is equal to the squares on BD, DA, for the angle at D is right.

— Elements, Book 2 Proposition 13

To substitute numeric values we can use Pythagorean triples, which are sets of three positive integers that satisfy the equation (Table 1).  To make this work, we’ll divide one of those sets by two.  This way we get numbers that are, if not whole, easy to work with.

Finally, we add the sets of numbers from Table 1 to Figure 2, and conclude with a re-statement of Book 2, Proposition 13.

From logical to numeric reasoning

In acute-angled triangles the square on the side subtending the acute angle is less than the squares on the sides containing the acute angle by twice the rectangle contained by one of the sides about the acute angle, namely that on which the perpendicular falls, and the straight line cut off within by the perpendicular towards the acute angle.

— Elements, Book 2 Proposition 13

Following along with Euclid’s argument, we have an acute triangle ABC, and have drawn a perpendicular from point A to point D.  (Figure 2). Observe that the perpendicular is a common side on two right triangles inside ABC, and that the value 12 is assigned to it.  (We have factored the table of Pythagorean triples to make it so.)

The proposition says that the square of side (AC) is less than the squares on the sides (CB) and (BA) by twice the value contained by (CB), (BD).  Or

(12.5)² is less than (8.5)² plus (13)² by 2 times (8.5 x 5)

Re-arranging terms

241.25 – 156.25 = 85

Therefore, in this acute triangle the square of the hypotenuse subtending angle B (156.25) is less than squares on the sides (241.25) by 2 x (8.5 x 5).

Q.E.D.

Appendix A: Meta Llama 3.1 B-Instruct (open-source)

“The square of the hypotenuse (156.25) is actually greater than the sum of the squares on the sides (241.25) by 85”. False.

Appendix B: Microsoft Phi 3.5 mini-instruct (open-source)

“… the sum of the squares on sides BD (8.5) and CB (13) by a factor of 2 times the product of the perpendicular (8.5)…” Orthographic errors. Incorrect numeric and label substitution.

Appendix C: OpenAI ChatGPT (online)

No issues

Appendix D: Google Gemini (online)

No issues