Mozilla QA & Test Automation

• Why Test Automation for Firefox?
• Buildbot and TBPL (Tinderboxpushlog)
• Mozilla QA & Firefox Releases
• Test Automation with Mozmill
• Mozmill CI system
• Q&A

As listed on http://www.ohloh.net/p/firefox:

• 10.7 million lines of code
• ~50.000 commits (last 12 months)
• 1100 contributors (last 12 months)

The way to keep up is Test Automation!

• Hybrid continuous integration system with annual >1M build jobs, >10M tests jobs, and >2.000 CPU years
• Various testsuites: XPCShell (X), Mochitests (M), Reftests (R), Marionette (Mn)

• Final versions released every 6 weeks
• Beta releases released twice a week

Responsibilities:

• Bug verifications for fixed bugs since the last beta release
• Exploratory testing, and regression testing based on testplans
• Triage for work items of the current iteration development cycle
• Observe automation results

• Test framework for functional tests (Firefox Desktop only)
• Used for tests, which cannot be run under buildbot
• Easy to execute and to get started for contribution
• Well, it's also not able to automate everything

Exclusive features:

• Ability to restart the application during a test
• Supports all available localized builds
• Allows network access for specific testruns

• Continuous integration driven by Jenkins and Mozilla Pulse
• 68 slave nodes, 17 platforms
• ~500 jobs/day for daily builds, ~650 jobs for releases

Slides: http://whimboo.github.io/slides/140426_berlin_hacking/

Contact: hskupin@mozilla.com

Home of the Firefox Automation team:

• https://wiki.mozilla.org/QA/Automation
• irc://irc.mozilla.org:6667/#automation