1 Introduction

Note

This is an EARLY DRAFT.

Artificial Intelligence (AI) refers to computer systems designed to perform tasks that typically require human intelligence, such as visual perception, speech recognition, decision-making, language translation, and problem solving. The field has evolved dramatically since its inception in the 1950s.

Machine learning (ML) represents a major paradigm within AI, focusing specifically on developing systems that can learn and improve from experience with data, rather than following explicitly programmed rules. While AI broadly encompasses the creation of intelligent behavior in machines, machine learning provides a data-driven approach to achieving this intelligence. This chapter will explore the evolution of AI and machine learning, examine the main types of machine learning approaches, and introduce the fundamental challenges in the field.

1.1 The AI Landscape

1.1.1 From Early AI to Machine Learning

While the term “artificial intelligence” was coined at a landmark 1956 Dartmouth conference, the quest to create thinking machines has much deeper roots. Early pioneers like Alan Turing, Warren McCulloch, and Walter Pitts had already laid crucial theoretical foundations in the 1940s. McCulloch and Pitts published their groundbreaking 1943 paper “A Logical Calculus of the Ideas Immanent in Nervous Activity,” which introduced the first mathematical model of a neural network. Their work showed how networks of simple artificial neurons could implement logical operations, suggesting that brain-like networks could perform complex computational tasks. This insight was revolutionary because it demonstrated, for the first time, how the brain’s biological structure might perform mathematical calculations. Their model proved influential in two major ways: it helped scientists better understand how the brain processes information, and it provided a blueprint for building artificial systems that could mimic brain-like computation. The early decades of formal AI research (1950s-1970s) focused on directly programming computers with explicit instructions and logical rules. Researchers pursued ambitious goals of creating computer systems that could think like humans. Early achievements included programs that could solve mathematical proofs and simulate basic conversations, but these systems were limited by their rigid, pre-programmed nature.

However, these rule-based systems showed significant limitations. While they worked well for problems with clear, unchanging rules like chess (as demonstrated by IBM’s Deep Blue beating Garry Kasparov in 1997), they struggled with more nuanced real-world tasks. For example, a rule-based medical diagnosis system would contain thousands of explicitly coded rules like “if a patient has fever AND cough AND chest pain, then check for pneumonia.” This approach proved inadequate for complex problems involving uncertainty or requiring adaptation.

1.1.2 The Rise of Machine Learning

The 1980s saw the emergence of machine learning as a distinct approach, with systems learning patterns directly from data rather than following pre-programmed rules. Key developments included:

The 1980s and 1990s saw steady progress in machine learning techniques and algorithms, though the field remained relatively niche. A major breakthrough came in the mid-2000s with new approaches to training artificial neural networks - computer systems loosely inspired by biological brains. These networks could learn increasingly complex patterns when trained on large amounts of data.

The field underwent a dramatic transformation in the 2010s, driven by three key factors. First, the increasing power and affordability of Graphics Processing Units (GPUs) made it practical to train much larger neural networks. Originally designed for video games, GPUs proved ideal for the parallel computations needed in machine learning. Second, the internet provided vast amounts of training data - images, text, videos and more. Third, new technical approaches helped overcome previous limitations in training deep neural networks.

This combination enabled remarkable achievements. In 2012, a system called AlexNet achieved unprecedented accuracy in recognizing objects in images, launching what became known as the deep learning revolution. In 2016, DeepMind’s AlphaGo system defeated world champion Lee Sedol at Go, a feat many experts thought decades away. The system learned strategies beyond human knowledge by playing millions of games against itself.

The last few years have seen even more dramatic advances. Large language models like GPT-3 can engage in human-like conversation, write creative fiction, and even generate working computer code. DeepMind’s AlphaFold system achieved a breakthrough in predicting protein structures, a fundamental challenge in biology. AI art tools like DALL-E 2 and Stable Diffusion can create stunning original images from text descriptions, while ChatGPT has demonstrated remarkable capabilities in natural language understanding and generation.

These systems represent a fundamental advance in artificial intelligence - moving from narrow, specialized tools to more general-purpose systems that can handle a broad range of tasks. They learn patterns from massive datasets, discovering subtle relationships that let them generalize to new situations. While still far from human-level general intelligence, they have already transformed fields from scientific research to creative arts.

Machine learning’s data-driven approach has several key advantages. The systems can capture subtle patterns that humans might not even be consciously aware of. They can adapt to changing situations as they’re exposed to new data. Most importantly, they can handle ambiguous real-world situations that don’t fit neatly into predefined rules. This is why machine learning has proven so successful for tasks like computer vision, speech recognition, and language translation - areas where traditional rule-based approaches struggled.

This evolution from explicitly programming rules to learning from data represents a fundamental change in how we approach artificial intelligence. While rule-based systems remain valuable for certain well-defined problems, machine learning has opened up new possibilities for tackling complex real-world tasks that were previously out of reach. The field continues to evolve rapidly, with new architectures and approaches emerging regularly, pushing the boundaries of what artificial intelligence can achieve.

1.2 Types of Machine Learning

Machine learning tasks generally fall into three main categories, each suited to different types of problems and data availability. Let’s explore each approach and its applications.

1.2.1 Supervised Learning

Supervised learning is a fundamental type of machine learning where we learn from examples that have been labeled with their correct answers. The term “supervised” comes from the idea that some knowledgeable entity (the supervisor) has already gone through the data and provided the right answer for each example. This supervisor could be:

Human experts manually labeling data (e.g., doctors marking which X-ray images show tumors)
Automated systems collecting natural labels (e.g., whether a user clicked on an ad)
Historical records with known outcomes (e.g., past house sales with their final prices)
Physical measurements (e.g., sensors recording the actual temperature)

The key is that for each training example, we have both the input features and the known correct output. Our task is then to learn patterns from these labeled examples that will let us make accurate predictions on new, unseen cases. Let’s explore some concrete examples that illustrate this paradigm.

Email spam detection is perhaps one of the most ubiquitous applications of supervised learning that we interact with daily. Gmail’s spam filter, for instance, analyzes various features of incoming emails - the sender’s address, email subject, content words, HTML structure, and embedded links. These features serve as input $X$, while the output $Y$ is a simple binary label: spam or not spam. The system learns from millions of emails that users have previously marked as spam or legitimate. This training data helps Gmail achieve remarkably high accuracy, with false positive rates below 0.1%. The filter continues to adapt as spammers evolve their tactics, learning from new examples of spam that users report.

House price prediction has become increasingly sophisticated with services like Zillow’s “Zestimate.” The system takes as input $X$ detailed property characteristics including square footage, number of bedrooms, location (often down to the exact GPS coordinates), age of the house, recent renovations, and even local school ratings. The output $Y$ is the predicted market value in dollars. Zillow trains its models on millions of actual home sales recorded in public databases. Their algorithms combine multiple prediction methods and regularly retrain on new sales data, achieving a median error rate of less than 2% in many major markets.

In medical diagnosis, systems like Stanford’s CheXNet analyze chest X-ray images ($X$) to detect various lung conditions ($Y$) like pneumonia, edema, and cardiomegaly. The model trains on large public datasets of labeled chest radiographs with clinically verified diagnoses. While not replacing human radiologists, these systems serve as valuable diagnostic aids, often matching or exceeding specialist-level accuracy.

Image classification has seen dramatic advances through deep learning, exemplified by systems like Google Photos. The input $X$ consists of raw pixel values from digital images, often millions of pixels per image. The system predicts category labels $Y$ such as “cat,” “dog,” “beach,” or “birthday party.” Modern image classifiers train on massive datasets like ImageNet, containing millions of labeled images. Google Photos can now recognize thousands of distinct objects and scenes with remarkable accuracy, even handling subtle categories like different dog breeds or architectural styles.

Text autocomplete has become remarkably sophisticated, as demonstrated by systems like Gmail’s Smart Compose and GitHub’s Copilot. These systems take as input $X$ the sequence of words or characters the user has already typed, along with surrounding context. The output $Y$ is a prediction of what the user will type next - from single words to entire sentences or code blocks. Training data comes from vast corpora of text: emails and documents for Smart Compose, or public code repositories for Copilot. These systems have become so effective that their suggestions are often indistinguishable from human-written text, though they can still make amusing mistakes that reveal their statistical nature.

1.2.2 Unsupervised Learning

In unsupervised learning, we work with unlabeled data, trying to discover hidden patterns or structures. Unlike supervised learning, there are no correct answers to guide the learning process - the algorithm must find meaningful patterns on its own.

Clustering algorithms find natural groupings in data by identifying items that are similar to each other. For example, in customer segmentation, retailers like Amazon analyze purchase histories, browsing behavior, and demographic data to group customers into distinct market segments. This helps them tailor recommendations and marketing strategies - one cluster might represent price-sensitive bargain hunters, while another captures luxury shoppers who prioritize quality over cost. In biology, clustering gene expression data helps researchers identify groups of genes that are activated together under different conditions, providing insights into biological pathways and disease mechanisms. Search engines use document clustering to organize millions of web pages into topically related groups, making it easier for users to explore related content.

Dimensionality reduction techniques transform complex high-dimensional data into simpler, lower-dimensional representations while preserving important patterns. Image compression is a familiar example - JPEG compression can reduce image file sizes by 10x or more by finding compact representations that capture the key visual information while discarding less important details. In machine learning, dimensionality reduction is often used for feature extraction, transforming raw data like pixel values or sensor readings into more meaningful features that help models learn more effectively. These techniques also enable visualization of complex datasets - for instance, projecting high-dimensional gene expression data onto 2D plots that biologists can interpret visually to identify patterns and relationships.

Anomaly detection algorithms learn what “normal” patterns look like in data and flag unusual deviations that may indicate problems or opportunities. Banks use these techniques to detect fraudulent credit card transactions by learning typical spending patterns for each customer - unusual purchases that deviate from these patterns trigger alerts for investigation. Network security systems monitor traffic patterns to detect potential cyber attacks or intrusions that don’t match normal network behavior. In manufacturing, anomaly detection helps with quality control by identifying subtle deviations in sensor readings that may indicate equipment problems or defective products before they cause major issues. The key advantage is that these systems can discover novel anomalies without being explicitly programmed to look for specific problems.

1.2.3 Reinforcement Learning

Reinforcement learning involves an agent learning to make sequences of decisions by interacting with an environment. Unlike supervised learning where we have correct answers, or unsupervised learning where we look for patterns, reinforcement learning focuses on learning optimal behavior through trial and error. The agent takes actions in an environment, receives feedback in the form of rewards or penalties, and gradually learns strategies that maximize its long-term rewards.

This process mirrors how humans and animals learn through experience. Just as a child learns to ride a bicycle through practice, falling down, and trying again, a reinforcement learning agent improves through repeated interaction with its environment. The agent must balance exploring new actions to discover better strategies with exploiting what it has already learned works well.

One of the most dramatic demonstrations of reinforcement learning’s potential was DeepMind’s AlphaGo system. The ancient game of Go had long been considered one of the greatest challenges for artificial intelligence due to its vast complexity - there are more possible board positions than atoms in the universe. While previous Go programs relied heavily on human expertise, AlphaGo learned largely through self-play, starting from basic rules and discovering sophisticated strategies on its own. The system would play millions of games against itself, receiving a reward when it won and a penalty when it lost. Through this process, it learned subtle patterns and principles that even human experts hadn’t recognized. This culminated in its historic victory over world champion Lee Sedol in 2016, including moves that commentators initially thought were mistakes but proved to be innovative winning strategies.

Other applications of reinforcement learning span diverse domains. In robotics, agents learn complex physical skills like walking or manipulating objects through repeated attempts and feedback. Data centers use reinforcement learning to optimize cooling systems, reducing energy usage while maintaining safe temperatures. Autonomous vehicles learn to navigate complex traffic scenarios by balancing multiple objectives like safety, efficiency, and passenger comfort. In each case, the power of reinforcement learning comes from its ability to discover effective strategies through experience, even in situations too complex for explicit programming.

::: {.callout-note} This is an **EARLY DRAFT**. ::: # Introduction Artificial Intelligence (AI) refers to computer systems designed to perform tasks that typically require human intelligence, such as visual perception, speech recognition, decision-making, language translation, and problem solving. The field has evolved dramatically since its inception in the 1950s. Machine learning (ML) represents a major paradigm within AI, focusing specifically on developing systems that can learn and improve from experience with data, rather than following explicitly programmed rules. While AI broadly encompasses the creation of intelligent behavior in machines, machine learning provides a data-driven approach to achieving this intelligence. This chapter will explore the evolution of AI and machine learning, examine the main types of machine learning approaches, and introduce the fundamental challenges in the field. ## The AI Landscape ### From Early AI to Machine Learning While the term "artificial intelligence" was coined at a landmark 1956 Dartmouth conference, the quest to create thinking machines has much deeper roots. Early pioneers like Alan Turing, Warren McCulloch, and Walter Pitts had already laid crucial theoretical foundations in the 1940s. McCulloch and Pitts published their groundbreaking 1943 paper "A Logical Calculus of the Ideas Immanent in Nervous Activity," which introduced the first mathematical model of a neural network. Their work showed how networks of simple artificial neurons could implement logical operations, suggesting that brain-like networks could perform complex computational tasks. This insight was revolutionary because it demonstrated, for the first time, how the brain's biological structure might perform mathematical calculations. Their model proved influential in two major ways: it helped scientists better understand how the brain processes information, and it provided a blueprint for building artificial systems that could mimic brain-like computation. The early decades of formal AI research (1950s-1970s) focused on directly programming computers with explicit instructions and logical rules. Researchers pursued ambitious goals of creating computer systems that could think like humans. Early achievements included programs that could solve mathematical proofs and simulate basic conversations, but these systems were limited by their rigid, pre-programmed nature. However, these rule-based systems showed significant limitations. While they worked well for problems with clear, unchanging rules like chess (as demonstrated by IBM's Deep Blue beating Garry Kasparov in 1997), they struggled with more nuanced real-world tasks. For example, a rule-based medical diagnosis system would contain thousands of explicitly coded rules like "if a patient has fever AND cough AND chest pain, then check for pneumonia." This approach proved inadequate for complex problems involving uncertainty or requiring adaptation. ### The Rise of Machine Learning The 1980s saw the emergence of machine learning as a distinct approach, with systems learning patterns directly from data rather than following pre-programmed rules. Key developments included: The 1980s and 1990s saw steady progress in machine learning techniques and algorithms, though the field remained relatively niche. A major breakthrough came in the mid-2000s with new approaches to training artificial neural networks - computer systems loosely inspired by biological brains. These networks could learn increasingly complex patterns when trained on large amounts of data. The field underwent a dramatic transformation in the 2010s, driven by three key factors. First, the increasing power and affordability of Graphics Processing Units (GPUs) made it practical to train much larger neural networks. Originally designed for video games, GPUs proved ideal for the parallel computations needed in machine learning. Second, the internet provided vast amounts of training data - images, text, videos and more. Third, new technical approaches helped overcome previous limitations in training deep neural networks. This combination enabled remarkable achievements. In 2012, a system called AlexNet achieved unprecedented accuracy in recognizing objects in images, launching what became known as the deep learning revolution. In 2016, DeepMind's AlphaGo system defeated world champion Lee Sedol at Go, a feat many experts thought decades away. The system learned strategies beyond human knowledge by playing millions of games against itself. The last few years have seen even more dramatic advances. Large language models like GPT-3 can engage in human-like conversation, write creative fiction, and even generate working computer code. DeepMind's AlphaFold system achieved a breakthrough in predicting protein structures, a fundamental challenge in biology. AI art tools like DALL-E 2 and Stable Diffusion can create stunning original images from text descriptions, while ChatGPT has demonstrated remarkable capabilities in natural language understanding and generation. These systems represent a fundamental advance in artificial intelligence - moving from narrow, specialized tools to more general-purpose systems that can handle a broad range of tasks. They learn patterns from massive datasets, discovering subtle relationships that let them generalize to new situations. While still far from human-level general intelligence, they have already transformed fields from scientific research to creative arts. Machine learning's data-driven approach has several key advantages. The systems can capture subtle patterns that humans might not even be consciously aware of. They can adapt to changing situations as they're exposed to new data. Most importantly, they can handle ambiguous real-world situations that don't fit neatly into predefined rules. This is why machine learning has proven so successful for tasks like computer vision, speech recognition, and language translation - areas where traditional rule-based approaches struggled. This evolution from explicitly programming rules to learning from data represents a fundamental change in how we approach artificial intelligence. While rule-based systems remain valuable for certain well-defined problems, machine learning has opened up new possibilities for tackling complex real-world tasks that were previously out of reach. The field continues to evolve rapidly, with new architectures and approaches emerging regularly, pushing the boundaries of what artificial intelligence can achieve. ## Types of Machine Learning Machine learning tasks generally fall into three main categories, each suited to different types of problems and data availability. Let's explore each approach and its applications. ### Supervised Learning Supervised learning is a fundamental type of machine learning where we learn from examples that have been labeled with their correct answers. The term "supervised" comes from the idea that some knowledgeable entity (the supervisor) has already gone through the data and provided the right answer for each example. This supervisor could be: - Human experts manually labeling data (e.g., doctors marking which X-ray images show tumors) - Automated systems collecting natural labels (e.g., whether a user clicked on an ad) - Historical records with known outcomes (e.g., past house sales with their final prices) - Physical measurements (e.g., sensors recording the actual temperature) The key is that for each training example, we have both the input features and the known correct output. Our task is then to learn patterns from these labeled examples that will let us make accurate predictions on new, unseen cases. Let's explore some concrete examples that illustrate this paradigm. Email spam detection is perhaps one of the most ubiquitous applications of supervised learning that we interact with daily. Gmail's spam filter, for instance, analyzes various features of incoming emails - the sender's address, email subject, content words, HTML structure, and embedded links. These features serve as input $X$, while the output $Y$ is a simple binary label: spam or not spam. The system learns from millions of emails that users have previously marked as spam or legitimate. This training data helps Gmail achieve remarkably high accuracy, with false positive rates below 0.1%. The filter continues to adapt as spammers evolve their tactics, learning from new examples of spam that users report. House price prediction has become increasingly sophisticated with services like Zillow's "Zestimate." The system takes as input $X$ detailed property characteristics including square footage, number of bedrooms, location (often down to the exact GPS coordinates), age of the house, recent renovations, and even local school ratings. The output $Y$ is the predicted market value in dollars. Zillow trains its models on millions of actual home sales recorded in public databases. Their algorithms combine multiple prediction methods and regularly retrain on new sales data, achieving a median error rate of less than 2% in many major markets. In medical diagnosis, systems like Stanford's CheXNet analyze chest X-ray images ($X$) to detect various lung conditions ($Y$) like pneumonia, edema, and cardiomegaly. The model trains on large public datasets of labeled chest radiographs with clinically verified diagnoses. While not replacing human radiologists, these systems serve as valuable diagnostic aids, often matching or exceeding specialist-level accuracy. Image classification has seen dramatic advances through deep learning, exemplified by systems like Google Photos. The input $X$ consists of raw pixel values from digital images, often millions of pixels per image. The system predicts category labels $Y$ such as "cat," "dog," "beach," or "birthday party." Modern image classifiers train on massive datasets like ImageNet, containing millions of labeled images. Google Photos can now recognize thousands of distinct objects and scenes with remarkable accuracy, even handling subtle categories like different dog breeds or architectural styles. Text autocomplete has become remarkably sophisticated, as demonstrated by systems like Gmail's Smart Compose and GitHub's Copilot. These systems take as input $X$ the sequence of words or characters the user has already typed, along with surrounding context. The output $Y$ is a prediction of what the user will type next - from single words to entire sentences or code blocks. Training data comes from vast corpora of text: emails and documents for Smart Compose, or public code repositories for Copilot. These systems have become so effective that their suggestions are often indistinguishable from human-written text, though they can still make amusing mistakes that reveal their statistical nature. ### Unsupervised Learning In unsupervised learning, we work with unlabeled data, trying to discover hidden patterns or structures. Unlike supervised learning, there are no correct answers to guide the learning process - the algorithm must find meaningful patterns on its own. Clustering algorithms find natural groupings in data by identifying items that are similar to each other. For example, in customer segmentation, retailers like Amazon analyze purchase histories, browsing behavior, and demographic data to group customers into distinct market segments. This helps them tailor recommendations and marketing strategies - one cluster might represent price-sensitive bargain hunters, while another captures luxury shoppers who prioritize quality over cost. In biology, clustering gene expression data helps researchers identify groups of genes that are activated together under different conditions, providing insights into biological pathways and disease mechanisms. Search engines use document clustering to organize millions of web pages into topically related groups, making it easier for users to explore related content. Dimensionality reduction techniques transform complex high-dimensional data into simpler, lower-dimensional representations while preserving important patterns. Image compression is a familiar example - JPEG compression can reduce image file sizes by 10x or more by finding compact representations that capture the key visual information while discarding less important details. In machine learning, dimensionality reduction is often used for feature extraction, transforming raw data like pixel values or sensor readings into more meaningful features that help models learn more effectively. These techniques also enable visualization of complex datasets - for instance, projecting high-dimensional gene expression data onto 2D plots that biologists can interpret visually to identify patterns and relationships. Anomaly detection algorithms learn what "normal" patterns look like in data and flag unusual deviations that may indicate problems or opportunities. Banks use these techniques to detect fraudulent credit card transactions by learning typical spending patterns for each customer - unusual purchases that deviate from these patterns trigger alerts for investigation. Network security systems monitor traffic patterns to detect potential cyber attacks or intrusions that don't match normal network behavior. In manufacturing, anomaly detection helps with quality control by identifying subtle deviations in sensor readings that may indicate equipment problems or defective products before they cause major issues. The key advantage is that these systems can discover novel anomalies without being explicitly programmed to look for specific problems. ### Reinforcement Learning Reinforcement learning involves an agent learning to make sequences of decisions by interacting with an environment. Unlike supervised learning where we have correct answers, or unsupervised learning where we look for patterns, reinforcement learning focuses on learning optimal behavior through trial and error. The agent takes actions in an environment, receives feedback in the form of rewards or penalties, and gradually learns strategies that maximize its long-term rewards. This process mirrors how humans and animals learn through experience. Just as a child learns to ride a bicycle through practice, falling down, and trying again, a reinforcement learning agent improves through repeated interaction with its environment. The agent must balance exploring new actions to discover better strategies with exploiting what it has already learned works well. One of the most dramatic demonstrations of reinforcement learning's potential was DeepMind's AlphaGo system. The ancient game of Go had long been considered one of the greatest challenges for artificial intelligence due to its vast complexity - there are more possible board positions than atoms in the universe. While previous Go programs relied heavily on human expertise, AlphaGo learned largely through self-play, starting from basic rules and discovering sophisticated strategies on its own. The system would play millions of games against itself, receiving a reward when it won and a penalty when it lost. Through this process, it learned subtle patterns and principles that even human experts hadn't recognized. This culminated in its historic victory over world champion Lee Sedol in 2016, including moves that commentators initially thought were mistakes but proved to be innovative winning strategies. Other applications of reinforcement learning span diverse domains. In robotics, agents learn complex physical skills like walking or manipulating objects through repeated attempts and feedback. Data centers use reinforcement learning to optimize cooling systems, reducing energy usage while maintaining safe temperatures. Autonomous vehicles learn to navigate complex traffic scenarios by balancing multiple objectives like safety, efficiency, and passenger comfort. In each case, the power of reinforcement learning comes from its ability to discover effective strategies through experience, even in situations too complex for explicit programming.