Nordic Testing Days 2025

How can you test a system whose answer changes every time? In AI, data is the spec: the only reliable quality gauge is how the model performs against a living dataset. This talk shows how applying that principle transforms flashy demos into dependable products.

Key Takeaways:

• What happens if you never go beyond vibe-checking for quality
• How to meaningfully evaluate quality of AI systems
• How datasets can align everyone about what "good" means

In this deep-dive technical session, I'll share how Uber revamped its mobile testing approach using large language models (LLMs) to create DragonCrawl - a system that tests mobile applications with human-like intuition. I'll walk through how we transformed mobile testing from a maintenance-heavy, script-based approach to an intelligent, adaptive system that can handle UI changes automatically across multiple languages and cities.

The session will cover:

• The challenges of traditional mobile testing at scale (3,000+ simultaneous experiments)
• Architecture and implementation of DragonCrawl using MPNet and embedding techniques
• Real-world examples of DragonCrawl's adaptive behavior and problem-solving capabilities
• Practical strategies for handling LLM challenges like hallucinations and adversarial cases
• Results and metrics from production deployment
• Live demonstration of DragonCrawl in action

Key Takeaways:

• Understanding how to leverage LLMs for automated testing, including model selection criteria, architecture decisions, and implementation strategies that enable human-like testing behavior
• Practical techniques for handling LLM challenges in production systems, including specific approaches to manage hallucinations, adversarial cases, and edge scenarios
• Real-world insights into scaling automated testing across multiple languages and locations without maintaining separate test scripts, including specific metrics and benchmarks from Uber's production environment

There's a reason people in tech are notorious for answering questions with questions like "what do you mean by X?" or "that depends".

Human language is kinda vague and sloppy, but context usually fills in the blanks, and we're used to be able to clarify on the spot, but that doesn't work when we try to tell machines what to do, or when when we're putting words in documents that other people in other contexts are supposed to understand. This is, in my experience, where bugs usually occur – in the assumed understanding of constructs - the conceptual "things" we use in language, like "hour", "border", "flower", "write", "red" and "sound". In other words, things go wrong when we think that everyone must mean the same thing, when we think something is clarified enough.

The first step in uncovering these bug-generating mistakes is becoming aware of them and getting creative about alternative meanings, and that's part of why it's so much fun being a tester.

In this talk I will present some of my "favourite" constructs to challenge, and some useful cues to help you notice that a construct you're using should be investigated closer.

Key Takeaways:

• A structured way of using construct validity as an attack vector for finding bugs, even without looking under the hood, like you had x-ray vision.
• A list of very common problematic constructs, and why they keep tripping people up, again and again.
• Some phrases you may choose, to come across as less annoying, when you actually care about what others dismissively refer to as "just semantics" .

Playwright stands out as a powerful tool, but what exactly sets it apart from other automation solutions? This talk aims to introduce attendees to the key differentiators and capabilities that make Playwright a superior choice for web testing.
 
We will begin by exploring the fundamental strengths of Playwright, such as its ability to handle browser interactions with remarkable speed and efficiency. However, the true power of Playwright lies in its advanced features that go beyond basic functionalities.
 
Throughout the session, we will delve into a carefully curated list of real-world use cases where Playwright has demonstrated its superiority over other automation tools. These examples will highlight how Playwright excels in testing complex web applications within large organizations, showcasing its robustness and versatility.
 
One of the standout features of Playwright is its support for authentication contexts. This allows testers to simulate different user sessions and roles seamlessly, making it easier to test applications that require user authentication. Additionally, storage states enable the preservation of cookies and local storage data between tests, ensuring consistency and reliability in test scenarios.
 
Another powerful capability of Playwright is its tracing feature, which provides detailed insights into test execution. This includes capturing screenshots, network activity, and console logs, making it easier to diagnose and debug issues.
 
By the end of this talk, attendees will have a comprehensive understanding of how Playwright's advanced features can enhance their testing experience. They will learn how these capabilities not only improve the efficiency and effectiveness of their tests but also make the overall testing process more enjoyable and productive for engineers.
 
Key Takeaways:

• Superior Web Automation Capabilities: Understand what sets Playwright apart from other web automation tools, including its high execution speed and efficient browser interactions.
• Advanced Features for Enhanced Testing: Learn about Playwright's powerful features like authentication contexts, storage states, and tracing, which significantly improve the testing experience and make it more efficient.
• Real-World Use Cases: Gain insights from real-world examples where Playwright has proven its superiority in testing complex web applications within large organizations.
• Improved Debugging and Diagnostics: Discover how Playwright's tracing feature provides detailed insights into test execution, helping to diagnose and debug issues more effectively.
• Comprehensive Testing Framework: By the end of the talk, attendees will be equipped with the knowledge to leverage Playwright's advanced capabilities, making their testing processes more robust and enjoyable.

Modernizing the testing of payment systems in the Nordic region’s largest bank is a complex journey in a highly regulated, legacy environment. With millions of daily transactions  and billions of euros going through our systems, ensuring fault tolerance and system reliability is critical — not just for the bank, but for society at large. As the financial sector faced increasing pressure from regulatory demands, the need to move from mainframes to modern software, and demands for faster time-to-market, we recognized that our current traditional QA approach was unsustainable and  not scalable for future demands.

This presentation will share our journey of transforming QA into a business enabler. We navigated significant hurdles, including securing senior management buy-in, managing high delivery pressures, and shifting mindsets from manual testing to automation-driven quality. By focusing on foundational improvements — such as evolving our test automation framework, updating our tool stack, including integrating CI/CD, optimizing test environments, and implementing fast technical checks — we achieved remarkable results.

We reduced test cycle times from three weeks to just three hours, increased automation coverage to 75% across all test levels, and improved production quality by minimizing incidents. This transformation accelerated software releases and strengthened trust across teams and leadership.

Join us to learn the critical lessons, challenges, and best practices for modernizing QA in a large-scale, complex financial ecosystem.

Key Takeaways:

• How to drive QA transformation in a highly regulated, high-stakes environment
• How shifting team culture and mindset is key to turning quality into a shared responsibility
• Defining first steps when modernizing QA

In a world where over 1 billion people live with disabilities, accessibility is no longer optional — it’s essential. The European Accessibility Act (EAA), coming into full effect in June 2025, sets a legal framework that requires businesses to provide accessible digital goods and services. This creates both a compliance mandate and an opportunity to unlock new markets, improve user experiences, and foster innovation.

We’ll explore the current challenges in accessibility testing, such as the limitations of automated tools, the importance of real-user feedback, and the complexities of testing emerging technologies like AR/VR. Learn from real-world examples, including how Accenture has partnered with clients to integrate accessibility into their digital ecosystems by integrating it into their design and development pipelines.

The talk will also introduce cutting-edge trends, such as AI-powered accessibility tools, inclusive design thinking, and the integration of accessibility into CI/CD workflows. Practical steps will be shared, including how to involve diverse users in testing and the critical role of an accessibility statement in building trust and driving continuous improvement.

Attendees will leave inspired to champion accessibility as a mindset and practice, equipped with actionable insights to create inclusive, user-friendly digital experiences that benefit everyone. Whether you’re a tester, developer, or decision-maker, this talk will show you how accessibility can be a catalyst for innovation and inclusion in your projects.

Key Takeaways:

• Accessibility Drives Innovation and Opportunity: Embrace accessibility to unlock new markets, improve user experiences, and future-proof your business.
• Think Beyond Compliance: Leverage cutting-edge tools and inclusive design to create meaningful, barrier-free digital experiences.
• Commit to Inclusivity: An accessibility statement isn’t just a policy—it’s a promise to your users and a tool for continuous improvement.

Being a good tester requires a diverse skill set, including analytical, technical and social skills. While some of these skills take time to develop, others may not be immediately recognized as necessary.

In my talk, after providing some context about what testing means in a regulated environment, I invite you to analyze five 'hidden skills' that I discovered for myself over 15 years of testing, integrating and validating safety-critical railway systems.

1. Train yourself to recognize and accept your biases and limitations as a tester.
2. Understand your true role in a team or project.
3. Grasp the essence of norms and processes in regulated environments to apply them correctly.
4. Good testers ask questions that often translate to "What is the actual risk?"
5. Changing perspectives and reframing situations is a powerful skill.

I will share personal experiences, both funny and serious, and provide tips on nurturing these skills. This talk offers relatable experiences filled with mistakes, successes and bloopers from the regulated and complex railway industry, which is not widely known to most conference attendees. Plus, I will be talking a lot about trains!

Key Takeaways:

• Gain insights into the importance of recognizing and overcoming your human biases and limitations.
• Understand the significance of acknowledging and accepting your true role in a team or project, which may differ from your job profile. Sometimes, you might take on a "diligent bee" role, while other times, you might become the "defender."
• Realize that norms and processes are designed to assist, not create unnecessary activities or artefacts to hide behind.

This keynote looks at the fine art of translating technical information into human language without dumbing things down or setting off the BS detector (your own or corporate). Whether you’re explaining that usability testing is not a security test, penetration testing is not a sexual act, that "it works in Dev" isn’t the end of the story, or just trying to get a sentence in without an existential crisis, communication is everything.

Liisa Past will explore how tech professionals can adjust their style, level of detail, and delivery to fit their audience, be it customers, managers, or cross-functional teams. In the end, the goal is the same: get them to decide, believe, and act in your favour. She’ll dive into what makes communication purposeful and persuasive: knowing what your audience needs to hear, how they need to hear it, and how to keep their attention long enough to actually care.

Expect tips you can use, a war story or two (some may involve engineers hiding in the bathroom hoping the humans will leave), and a healthy dose of humour. One of her tattoos might be used as a legitimate prop.