Running Flutter integration tests on Firebase Test Lab

Photo by Daniel Romero on Unsplash

Hi! Today I’d like to show you how to run Flutter integration tests on devices using Firebase Test Lab.

What are integration tests?

Integration tests check how different parts of the app work together. Unlike unit tests, they are intended to be run on devices (physical or virtual) and are used for end-to-end testing instead of testing particular components in separation. Because of that, integrations tests are much slower than unit tests, but they are a great way to check if our app works correctly in real-life scenarios.

What is a Firebase Test Lab?

While running integrations tests is a great way of testing the app, we’re unfortunately limited to the devices we have. They can be either physical devices, emulators or simulators, but we have to setup each one of them to run tests on our machine. That can become problematic if we’d like to test it on many different devices.

With Firebase Test Lab, we have access to thousands of Google-hosted devices with different device configurations. They are all stored in a cloud, so we can have remote access to them. All we have to do is to upload our app’s tests and Firebase will run them for us.

Pre-requisites

For this example, I’m going to use the default counter app that’s generated when you run flutter create command.

If you want to follow these steps along the way, you can check out this repository’s branch: flutter_integration_testing

Setup

We’re gonna start with adding a dependency in the pubspec.yaml to dev_dependencies section. That’s the section where all dependencies for non-production code should be put.

We’ll use integration_test package. To install it, add this to your pubspec.yaml:

dev_dependencies:
  flutter_test:
    sdk: flutter
  integration_test: ^1.0.2+2

Add integrations tests support

To add support for integration testing, the first thing to do is to add method IntegrationTestWidgetsFlutterBinding.ensureInitialized() in your test. This method is needed to initialize the Flutter test environment.

In my case, I’m going to add it in the test/widget_test.dart file. That’s the basic widget test added to the counter app on start that checks if the text is updated on the screen whenever we tap ‘+’ button:

Full source code: widget_test.dart

Running test on your machine

To run the test on your machine, you have to first create an entry point for test driver script. To do that, create a new file in test_driver/integration_test.dart and paste it:

Full source code: integration_test.dart

That file can then be used to run flutter drive command and run all tests from test/widget_integration_test.dart. Remember that flutter drive runs test on your device, so you should have one already running!

flutter drive \
 --driver=test_driver/integration_test.dart \
 --target=test/widget_integration_test.dart

As the result, you should see that your app is installed, tests are running quickly and later on you’ll see test results:

Running tests on Firebase Test Lab

If you haven’t already done it, you can start with creating new Firebase project in the console. You don’t have to setup Android & iOS apps, you can finish when you see that screen:

Now you can go and setup either Android, iOS, or both apps and prepare them to upload to Firebase Test Lab.

Note: This article describes uploading Android versions. If you’d like to see how to setup iOS, check out this integration_test package documentation.

Android setup

To setup Android, you have to create a new file in android/app/src/androidTest/java/your/package/yourapp/MainActivityTest.java directory. That /java/your/package/yourapp path should be replaced with your path:

Full source code: MainActivityTest.java

And the last thing is to add dependencies for android testing in android/app/build.gradle file:

Full source code: build.gradle

After this setup, you can prepare the instrumentation test build with the following command:

pushd android
flutter build apk
./gradlew app:assembleAndroidTest
./gradlew app:assembleDebug -Ptarget=test/widget_test.dart
popd

That should generate file in /build/app/outputs/apk/debug/app-debug.apk. Take that file and drag it (or upload) into the Firebase console in the Test Lab tab:

This will automatically start Robo testing. That’s another test type that you can explore, but we’re not gonna focus on that today. Instead, we’ll go to running our integration tests.

To do that, click on “Run a test” and choose “Instrumentation”:

On the next screen you should upload two previously generated APKs:

1. App APK is located in /build/app/outputs/apk/debug/app-debug.apk
2. Test App is located in /build/app/outputs/apk/androidTest/app-debug-androidTest.apk

After they are uploaded, on the next screen you can choose on which devices you want to run integration tests:

After selecting devices and continuing, your tests will start to run. After a few minutes you’ll see the results. If everything is ok you should see this screen with green success!

If you’ll have some test failures, you’ll see them here too.

And that’s it! If you want to check out the full source code, visit my repository:

pszklarska/flutter_integration_testing

I hope you liked this article. If you want to read more of my articles, don’t forget to follow me. Stay tuned for more! 🙌