AI Engineering from Scratch: Build Your First AI Application

Learn the foundations of AI engineering and build a small application powered by a pretrained language model. Begin with essential Python and model concepts, then practice prompting, retrieval, evaluation, and safe application design. Complete a guided capstone that brings these skills together, with balanced explanations and hands-on exercises designed for beginners.

Level: beginner · 24 lessons · about 8 h · Free · AI-assisted course created by Sandip Das

What you will learn

• Explain how AI, machine learning, and generative AI relate and when to use pretrained models.
• Read and adapt basic Python code that sends model requests and handles responses.
• Build a small language-model application using prompts and structured outputs.
• Add simple retrieval from a small document collection to ground model responses.
• Evaluate application outputs with a small test set for relevance, grounding, and failure cases.
• Apply basic safeguards for secrets, sensitive data, prompt injection, latency, and cost.
• Assemble and demonstrate a guided AI application capstone with documented limitations.

Course outline

1. Understand AI Application Engineering

Build a beginner's mental model of AI and choose a small, realistic document-question-answering project.

1. From AI to Generative AI (20 min) · free preview
   Use everyday examples and a relationship diagram to distinguish learning from data, generation, and ordinary programmed rules.
2. Define a Small, Useful AI Application (20 min)
   Map inputs, model calls, outputs, and human checks for a local document-grounded question-answering application.

2. Python Essentials and Your First Request

Set up an isolated project and learn just enough Python to send, inspect, and safely handle a model request.

1. Set Up Python Without Exposing Secrets (20 min)
   Create a virtual environment, install the selected dependencies, and configure credentials outside source code; offer a clearly labeled no-paid-API replay path.
2. Read Python Values, Lists, and Dictionaries (20 min)
   Explain code line by line and practice representing questions, passages, and answers with beginner-friendly data structures.
3. Use Functions, Files, and JSON (20 min)
   Build small reusable functions to load a text file, read JSON records, and handle a missing file without a crash.
4. Send and Inspect Your First Model Request (20 min)
   Trace request and response data through a minimal SDK example, distinguishing actual API execution from offline replay.

3. Model Behavior and Effective Prompts

Understand token budgets and model limitations, then improve prompts through small controlled comparisons.

1. Understand Tokens, Context, and Inference (20 min)
   Use a request-flow diagram and response usage to explain how pretrained models generate text and consume finite context capacity.
2. Write Prompts That State the Task Clearly (20 min)
   Revise a vague prompt into explicit application instructions, separately labeled user data, and a small example.
3. Choose Models and Recognize Uncertainty (20 min)
   Practice spotting unsupported answers and compare quality, latency, and token use without treating generation settings as universal.

4. Turn Model Text into Checked Application Data

Introduce lightweight typed output contracts and explicit handling of invalid, refused, or incomplete responses.

1. Define an Answer Schema (20 min)
   Connect familiar JSON dictionaries to a minimal, fully explained Pydantic model for answers, evidence labels, and an insufficient-information flag.
2. Request Structured Output and Handle Failures (20 min)
   Integrate structured parsing into the request function and keep failure paths distinct from successful answers.

5. Retrieve Evidence from a Small Document Collection

Prepare a safe local corpus, create traceable chunks, and retrieve passages using a lightweight embedding-based search.

1. Load and Chunk Safe Documents (20 min)
   Use a few public or synthetic plain-text documents, preserving source labels while splitting them into coherent passages.
2. Find Similar Passages with Embeddings (20 min)
   Explain vectors with a simple diagram and adapt a small local embedding-search scaffold without a database or cloud dependency.
3. Check Retrieval Before Generating Answers (20 min)
   Compare exact-term and paraphrased questions, inspect missed evidence, and change one retrieval setting at a time.

6. Connect Retrieval to Grounded Answers

Assemble retrieval, evidence-aware prompts, structured answers, citations, and a local interactive loop.

1. Build the Retrieve-Then-Answer Pipeline (20 min)
   Connect the search function to a prompt with separately labeled instructions, passages, and question.
2. Add Citations and Insufficient-Evidence Responses (20 min)
   Extend the output contract with source labels and check answers against the actual retrieved passages rather than trusting formatting.
3. Run a Local Question-Answering Application (20 min)
   Combine the functions into a minimal command-line loop with clear evidence display and graceful failures.

7. Evaluate Quality and Find Failure Causes

Use a small reusable test set and simple human-readable rubrics instead of relying on a persuasive demonstration.

1. Create a Small Test Set and Scoring Rubric (20 min)
   Prepare answerable, paraphrased, ambiguous, and unanswerable questions with expected passages and scoring anchors.
2. Run Tests, Locate Failures, and Retest (20 min)
   Record retrieval and answer scores separately, investigate a failure, and compare a single revision against the baseline.

8. Add Safeguards and Operating Limits

Harden the beginner application against data leaks and malicious document instructions while bounding failures, latency, and spending.

1. Protect Data and Treat Retrieved Text as Untrusted (20 min)
   Test direct and document-borne instruction attacks using synthetic data, and apply data minimization, safe logging, and least-access principles.
2. Bound Errors, Cost, and Latency (20 min)
   Add explicit request limits and safe diagnostics, then sketch deployment responsibilities without building production infrastructure.

9. Finish and Demonstrate Your Document QA Capstone

Use the accumulated application and test artifacts to complete a reproducible local project, verify its behavior, and document honest limitations.

1. Assemble a Reproducible Local Project (20 min)
   Consolidate the incremental project rather than start over, with a small approved corpus, configuration example, dependency instructions, and run command.
2. Verify the Capstone Against Its Acceptance Criteria (20 min)
   Run the existing rubric and safety checks, record real observed results, and fix or document one remaining weakness.
3. Demonstrate Results and Document Limitations (20 min)
   Present a short local demonstration, a practical runbook and cheat sheet, and a reflection using review and interview-style questions.

Start the course on LearnXops