Training course · RAG & LLM Applications · Test Automation & QA

RAG & LLM Applications for Test Automation and QA

Develop RAG and LLM applications: embeddings, vector databases, chunking, retrieval, evaluation, cost control and data protection.

Duration
2 days (14 hrs)
Format
Online or on-site
Language
German or English
Participants
open dates or exclusive for your team
Certificate
Certificate of attendance

Overview

“RAG & LLM Applications for Test Automation and QA” is a hands-on course for roles such as Test automation engineers and SDETs, QA engineers and test managers and Developers responsible for test suites. Rather than learning RAG & LLM Applications in the abstract, you work on tasks from your daily routine – with examples, exercises and templates from Test Automation & QA.

Retrieval-augmented generation connects language models to your own knowledge: manuals, contracts, tickets, wikis or product data are made searchable and handed to the model as context at runtime. The result is assistants that give grounded answers with citations instead of hallucinating. In this two-day course you build a complete RAG pipeline, from document preparation through embeddings and a vector database to answer generation, and you learn which levers actually determine quality.

Day two takes you from prototype to robust application. You compare chunking strategies, combine semantic and lexical search, apply reranking, metadata filters and query rewriting, and measure the effect with systematic evaluations. You also cover prompt caching, cost control, latency optimisation, document-level access control and privacy-compliant operation with an eye on GDPR and the EU AI Act. By the end you know when classic RAG is enough, when a large context window or agentic retrieval is the better option, and how to justify your architecture choices.

Quality assurance is under pressure: release cycles are getting shorter, applications more complex, and AI-assisted coding produces more code in less time – all of which needs testing. Test automation engineers and QA professionals therefore benefit particularly from AI assistants and agents. These tools derive test cases from user stories and acceptance criteria, write Playwright or Cypress tests with robust selectors, generate API tests from OpenAPI specifications, analyse flaky tests and build test data generators. At the same time the role is shifting: QA becomes the function that systematically safeguards AI-generated code too. This course teaches you to use AI for test design, automation and failure analysis and to adapt your test strategy to the new pace of development.

The course is delivered live – either online in a virtual classroom or on-site in our training rooms in Munich, Salzburg, Berlin, Augsburg, Hanover, Bremen, Stuttgart and Kaiserslautern. We can also run it exclusively for your team.

Who is this course for?

  • Test automation engineers and SDETs
  • QA engineers and test managers
  • Developers responsible for test suites
  • Business testers moving into automation

Typical challenges we address

  • Unstable end-to-end tests with timing issues and fragile selectors
  • High effort for test case creation, maintenance and test data
  • Gaps in coverage of APIs, edge cases and failure scenarios
  • Growing volumes of AI-generated code that need targeted safeguarding

What you will be able to do

  • You build a complete RAG pipeline from document preparation to answers with citations.
  • You choose embedding model, chunking strategy and vector database on solid grounds.
  • You improve retrieval quality with hybrid search, reranking and metadata filters.
  • You measure application quality with systematic evaluations and detect regressions.
  • Derive test cases systematically from requirements with AI and automate them in Playwright, Cypress or API tests.
  • Analyse flaky tests and coverage gaps with AI support and fix them for good.
  • Adapt your test strategy to AI-assisted development and safeguard generated code in a targeted way.

Course outline

Day 1

Module 1 Foundations of RAG and LLM applications
  • Anatomy of a RAG pipeline: ingestion, indexing, retrieval, generation
  • Understanding embeddings: semantic similarity, dimensions, model selection
  • Weighing RAG against fine-tuning and long context windows
  • Structured outputs, citations and dealing with hallucinations
Module 2 Document preparation and chunking
  • Reliably extracting PDFs, Office documents, HTML and tables
  • Chunking strategies: fixed size, structure-aware, semantic, with overlap
  • Enriching metadata and generating contextual summaries per chunk
  • Incremental updates and deletion of documents in the index
Module 3 Vector databases and retrieval strategies
  • Comparing vector databases and extensions such as pgvector, Qdrant, Weaviate or OpenSearch
  • Hybrid search combining vector and full-text search with BM25
  • Reranking, metadata filtering and query rewriting
  • Agentic retrieval: letting the model decide on search steps and tools
Module 4 Evaluation and quality assurance
  • Building and maintaining test sets from real user questions
  • Retrieval metrics such as recall and precision, and answer metrics such as faithfulness
  • Using and calibrating LLM-as-a-judge sensibly
  • Regression testing when chunking, model or prompt changes

Day 2

Module 5 Operations, cost and data protection
  • Prompt caching, batch processing and model choice to cut costs
  • Optimising latency and streaming responses
  • Document-level permissions, tenant isolation and protection against prompt injection
  • Data protection: data residency, processing agreements, deletion policies, GDPR and the EU AI Act
Module 6 Practice lab: Test Automation & QA
  • Deriving test cases from requirements
  • Writing Playwright and Cypress tests
  • Analysing flaky tests
  • API tests from specifications
  • Generating test data
  • Coverage and mutation testing
Module 7 Data protection, quality & transfer to daily work
  • Test data should be synthetic or effectively anonymised, as production data containing personal information must not be used unprotected in test environments or prompts (GDPR). Credentials for test systems belong in secret stores rather than test code, and in regulated sectors test results must be traceable and documented, for example as evidence under NIS2 or the Cyber Resilience Act.
  • Ground rules for safe AI use in your team
  • Checking results: facts, sources, tests and the four-eyes principle
  • Personal action plan for the next 30 days
  • Open questions, outlook and resources for further learning

Practical scenarios from your field

Deriving test cases from requirements

The assistant turns user stories and acceptance criteria into structured test cases with equivalence classes, boundary values and negative scenarios, optionally as Gherkin scenarios for behaviour-driven development.

Writing Playwright and Cypress tests

The AI writes end-to-end tests with role-based selectors, page objects or fixtures and proper waiting instead of fixed delays, and runs them iteratively until they are stable.

Analysing flaky tests

The agent examines test history, traces, screenshots and logs, identifies causes such as race conditions, test interdependencies or unstable test data, and proposes lasting fixes.

API tests from specifications

OpenAPI descriptions are turned into API tests for status codes, schemas, authentication and error cases, plus contract tests that surface interface changes between teams early.

Generating test data

The AI builds generators for realistic synthetic test data, factories and seed scripts that deliberately cover edge cases without relying on real personal data. Privacy issues in test environments are avoided.

Coverage and mutation testing

Coverage reports and mutation testing results are analysed, untested paths identified and missing tests added in a targeted way to increase the real value of the suite.

Possible hands-on projects

Flaky test report

A script that merges JUnit XML results from multiple CI runs, detects unstable tests and produces a prioritised list with failure patterns.

Test data generator

A CLI that creates realistic synthetic records, including edge cases, from a database schema or JSON schema. Data volume and distribution are fully configurable.

E2E suite for a demo app

A Playwright test suite with page objects, visual comparisons and CI integration, built by an AI agent for an existing web application.

Prerequisites

Solid programming skills in Python (or TypeScript as an alternative), basic knowledge of databases and REST APIs, and some experience with LLM APIs. Experience in software testing or test automation and a working knowledge of at least one programming language such as JavaScript, TypeScript, Python or Java.

Technical requirements

A laptop with Python, Git, Docker and a code editor, plus API access to at least one LLM and embedding provider. Credentials for the exercises can be provided if needed.

Data protection & compliance

Test data should be synthetic or effectively anonymised, as production data containing personal information must not be used unprotected in test environments or prompts (GDPR). Credentials for test systems belong in secret stores rather than test code, and in regulated sectors test results must be traceable and documented, for example as evidence under NIS2 or the Cyber Resilience Act.

Suggested dates

All dates are suggestions – your preferred date is always possible.

  • Thu, 26 November 2026 – Fri, 27 November 2026 Online Request
  • Tue, 8 December 2026 – Wed, 9 December 2026 Berlin Request
  • Wed, 16 December 2026 – Thu, 17 December 2026 Online Request
  • Mon, 11 January 2027 – Tue, 12 January 2027 Stuttgart Request
  • Mon, 18 January 2027 – Tue, 19 January 2027 Online Request
  • Thu, 28 January 2027 – Fri, 29 January 2027 Hanover Request

Frequently asked questions

When is RAG better than fine-tuning?

RAG is the right choice when a model needs access to current, extensive or frequently changing knowledge and should cite its sources. Fine-tuning mainly changes style, format or specialised behaviour. In many projects RAG is the faster, cheaper and more controllable route, sometimes complemented by fine-tuning for niche tasks.

Which vector database should we use?

That depends on data volume, existing infrastructure, filtering needs and operating model. If you already run PostgreSQL, pgvector often takes you a long way, while specialised systems shine with large datasets and hybrid search. In the course you compare several options hands-on and derive criteria for your own decision.

How can RAG be run in a GDPR-compliant way?

Key factors are choosing providers with suitable data residency and a data processing agreement, enforcing document-level permissions, having a deletion policy for the index and logs, and minimising personal data. For highly sensitive data, self-hosted models are also an option. We work through these choices using realistic architectures.

Do I need programming skills for this course?

A working knowledge of a programming language helps, since you should be able to read, understand and adapt generated tests. Business testers with little coding experience still benefit greatly, because the AI explains code and supports the move into automation. The exercises are designed to accommodate different levels of prior knowledge.

Will AI replace manual and exploratory QA?

No. AI automates recurring tasks such as test case creation, scripting and failure analysis, freeing up time for exploratory testing, risk assessment and acceptance. With AI-generated code in particular, the critical judgement of experienced QA professionals becomes more important, not less.

How much does “RAG & LLM Applications for Test Automation and QA” cost?

The course costs a flat €949 plus VAT per participant – whether online or on-site. This includes 2 training day(s) with a live trainer, course materials, a lab environment and a certificate of attendance.

Can I suggest my own date?

Yes. Our suggested dates are just a guide. In the request form you can always describe your preferred date or period in free text – we check availability and confirm within one working day.

Is the course online or on-site?

Both are possible: live online in a virtual classroom or on-site in Munich, Salzburg, Berlin, Augsburg, Hanover, Bremen, Stuttgart or Kaiserslautern. We can also run the course exclusively for your team.

More courses for Test Automation and QA

More RAG & LLM Applications courses

All 29 RAG & LLM Applications courses

RAG & LLM Applications · MunichRAG & LLM Applications · SalzburgRAG & LLM Applications · BerlinRAG & LLM Applications · AugsburgRAG & LLM Applications · HanoverRAG & LLM Applications · BremenRAG & LLM Applications · StuttgartRAG & LLM Applications · KaiserslauternRAG & LLM Applications · Online