Integrating Patrol with flutter_gherkin (v3)

[nonvachara-paloit]

Hi.

I've been trying to integrate Patrol with flutter_gherkin.

So far, I can expose this by nativeAutomator to Gherkin's step definition by using custom World, like so.

class MyWorld extends FlutterWorld {
  late final NativeAutomator nativeAutomator;
  MyWorld() {
    nativeAutomator = NativeAutomator(config: nativeAutomatorConfig);
  }
  void init() async {
    await nativeAutomator.configure();
  }
}

Future<MyWorld> worldCreator(TestConfiguration config) async {
  var world = MyWorld();
  world.init();
  return world;
}

After that, I can use it, for example:

StepDefinitionGeneric appLoad() {
  return given1<String, MyWorld>(
    "{string} loaded",
    (key, context) async {
      find.byValueKey(key);
      context.world.nativeAutomator.grantPermissionWhenInUse();
    },
  );
}

Here're some of my questions.

1. I can't figure out how to setup hostAutomator yet as PatrolBinding.ensureInitialized() failed.
2. Do you have any concern over this approach?

Thank you for this awesome project!
Non.

[bartekpacia]

Hi @nonvachara-paloit, this looks great!

I can't figure out how to setup hostAutomator yet as PatrolBinding.ensureInitialized() failed.

Make sure your test_driver/integration_test.dart looks like this:

import 'package:patrol/patrol_driver.dart';

Future<void> main() => patrolIntegrationDriver();

Let me know if this helps you :)

[nonvachara-paloit]

Hi @bartekpacia .

Unfortunately, not yet.

Here's my test_driver/integration_test.dart

import "package:integration_test/integration_test_driver.dart"
    as integration_test_driver;
import "package:patrol/patrol_driver.dart";

Future<void> main() {
  // The Gherkin report data send back to this runner by the app after
  // the tests have run will be saved to this directory
  integration_test_driver.testOutputsDirectory =
      "integration_test/gherkin/reports";
  
  return patrolIntegrationDriver(
    timeout: const Duration(minutes: 90),
  );
}

And this's my updated MyWorld.

class MyWorld extends FlutterWorld {
  late final HostAutomator hostAutomator;
  late final NativeAutomator nativeAutomator;
  MyWorld() {
    final binding = PatrolBinding.ensureInitialized();
    //
    hostAutomator = HostAutomator(
      config: const HostAutomatorConfig(),
      binding: binding,
    );

    nativeAutomator = NativeAutomator(config: nativeAutomatorConfig);
  }
  void init() async {
    await nativeAutomator.configure();
  }
}

The (redacted) error is somewhat like this.

	══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
	The following assertion was thrown running a test:
	'package:flutter/src/foundation/binding.dart': Failed assertion: line 146 pos 12:
	'_debugInitializedType == null': is not true.
	When the exception was thrown, this was the stack:
	#2      new BindingBase (package:flutter/src/foundation/binding.dart:146:12)
       ...... the reset of the stack .....

I could come up with minimum, reproduce-able project, and create an issue later.
Much appreciates. 🙏

[bartekpacia]

Okay. Can you show me your integration_test/gherkin_suite_test.dart?

[nonvachara-paloit]

Gladly. But most of them are from flutter_gherkin "Get going" section, which involves code generation. In my case, gherkin_suite_test.g.dart was generated.

gherkin_suite_test.dart

@GherkinTestSuite()
void main() {
  executeTestSuite(
    configuration: FlutterTestConfiguration(
      reporters: [
        StdoutReporter(MessageLevel.error)
          ..setWriteLineFn(print)
          ..setWriteFn(print),
        ProgressReporter()
          ..setWriteLineFn(print)
          ..setWriteFn(print),
        TestRunSummaryReporter()
          ..setWriteLineFn(print)
          ..setWriteFn(print),
      ],
      stepDefinitions: [
        appLoad(),
      ],
      createWorld: (TestConfiguration config) => worldCreator(config),
    ),
    appMainFunction: (World world) => app.main(),
  );
}

gherkin_suite_test.g.dart

// GENERATED CODE - DO NOT MODIFY BY HAND

part of 'gherkin_suite_test.dart';

// **************************************************************************
// GherkinSuiteTestGenerator
// **************************************************************************

class _CustomGherkinIntegrationTestRunner extends GherkinIntegrationTestRunner {
  _CustomGherkinIntegrationTestRunner({
    required FlutterTestConfiguration configuration,
    required StartAppFn appMainFunction,
    required Timeout scenarioExecutionTimeout,
    AppLifecyclePumpHandlerFn? appLifecyclePumpHandler,
    LiveTestWidgetsFlutterBindingFramePolicy? framePolicy,
  }) : super(
          configuration: configuration,
          appMainFunction: appMainFunction,
          scenarioExecutionTimeout: scenarioExecutionTimeout,
          appLifecyclePumpHandler: appLifecyclePumpHandler,
          framePolicy: framePolicy,
        );

  @override
  void onRun() {
    testFeature0();
  }

  void testFeature0() {
    runFeature(
      name: 'Example test:',
      tags: <String>[],
      run: () {
        runScenario(
          name: 'Open App',
          description: null,
          path:
              '/xxxx/integration_test/features/example.feature',
          tags: <String>[],
          steps: [
            (
              TestDependencies dependencies,
              bool skip,
            ) async {
              return await runStep(
                name: 'Given "xx" loaded',
                multiLineStrings: <String>[],
                table: null,
                dependencies: dependencies,
                skip: skip,
              );
            },
... // some other test steps was removed. Basically, they're almost the same on the above one
          ],
          onAfter: () async => onAfterRunFeature(
            name: 'Example test',
            path:
                '/xxx/./integration_test/features/example.feature',
            description: null,
            tags: <String>[],
          ),
        );
      },
    );
  }
}

void executeTestSuite({
  required FlutterTestConfiguration configuration,
  required StartAppFn appMainFunction,
  Timeout scenarioExecutionTimeout = const Timeout(const Duration(minutes: 10)),
  AppLifecyclePumpHandlerFn? appLifecyclePumpHandler,
  LiveTestWidgetsFlutterBindingFramePolicy? framePolicy,
}) {
  _CustomGherkinIntegrationTestRunner(
    configuration: configuration,
    appMainFunction: appMainFunction,
    appLifecyclePumpHandler: appLifecyclePumpHandler,
    scenarioExecutionTimeout: scenarioExecutionTimeout,
    framePolicy: framePolicy,
  ).run();
}

[bartekpacia]

I'm not familiar with Gherkin (and the flutter_gherkin package) at all, and the order in which its methods are called. But the root cause is that something is probably calling IntegrationTestWidgetsFlutterBinding.ensureInitialized() or WidgetsFlutterBinding.ensureInitialized() before your MyWorld() constructor calls PatrolBinding.ensureInitialized(). Maybe worldCreator is doing this? Or maybe flutter_gherkin calls this somewhere under the hood? I don't know :(

I think at this point it'll be the faster if you create a repo reproducing this issue :)

Thanks a lot for your work!

[nonvachara-paloit]

Agree. I'll probably play around some more and try to fix it.
Then I'll either create an issue or post a working solution here. Depends on the outcome. 😂

Anyway, will keep you posted. Thank for your help 😁

By the way, Gherkin is neat. Check them out.

[bartekpacia]

Sure, good luck :)

By the way, Gherkin is neat. Check them out.

Yeah, I've also heard some good things about it. If only I had some time to play around with it, I'd do it!

[nonvachara-paloit]

I managed to make hostAutomator works. Here're my working code and step-by-step instructions.

1. Follow both Patrol and flutter_gherkin (v3-rc17, at the time of writing) normal dev_dependencies modification.
   pubspec.yaml

   dev_dependencies:
     integration_test:
       sdk: flutter
     flutter_gherkin: 3.0.0-rc.17
     build_runner: any
     patrol: ^0.7.6

2. Create integration_test/my_world.dart. We will have access to Patrol's HostAutomator and NativeAutomator from this class.

   import "package:flutter_gherkin/flutter_gherkin.dart";
   import "package:flutter_test/flutter_test.dart";
   import "package:gherkin/gherkin.dart";
   import "package:patrol/patrol.dart";
   
   Future<MyWorld> worldCreator(TestConfiguration config,
       NativeAutomator nativeAutomator, HostAutomator hostAutomator) async {
     var world =
         MyWorld(nativeAutomator: nativeAutomator, hostAutomator: hostAutomator);
     return world;
   }
   
   class MyWorld extends FlutterWorld {
     final HostAutomator hostAutomator;
     final NativeAutomator nativeAutomator;
     MyWorld({required this.nativeAutomator, required this.hostAutomator});
   }

3. Create test_driver/integration_test.dart with the following code. Put in your own stepDefinitions.

   import "package:integration_test/integration_test_driver.dart"
       as integration_test_driver;
   import "package:patrol/patrol_driver.dart";
   
   Future<void> main() {
     // The Gherkin report data send back to this runner by the app after
     // the tests have run will be saved to this directory
     integration_test_driver.testOutputsDirectory = "integration_test/reports";
   
     return patrolIntegrationDriver(
       timeout: const Duration(minutes: 90),
     );
   }

4. Create integration_test/gherkin_suite_test.dart

   import "package:flutter_gherkin/flutter_gherkin.dart";
   import "package:flutter_test/flutter_test.dart";
   import "package:gherkin/gherkin.dart";
   import "package:patrol/patrol.dart";
   // The application under test.
   import "package:example_app/main.dart" as app;
   
   import "config.dart";
   import "my_world.dart";
   import "steps/app_load.dart";
   import "steps/permission.dart";
   import "steps/screenshot.dart";
   
   part "gherkin_suite_test.g.dart";
   
   @GherkinTestSuite()
   void main() {
     final binding = PatrolBinding.ensureInitialized();
     final hostAutomator = HostAutomator(
       config: hostAutomatorConfig,
       binding: binding,
     );
     final nativeAutomator = NativeAutomator(config: nativeAutomatorConfig);
     nativeAutomator.configure();
     executeTestSuite(
       configuration: FlutterTestConfiguration(
         reporters: [
           StdoutReporter(MessageLevel.error)
             ..setWriteLineFn(print)
             ..setWriteFn(print),
           ProgressReporter()
             ..setWriteLineFn(print)
             ..setWriteFn(print),
           TestRunSummaryReporter()
             ..setWriteLineFn(print)
             ..setWriteFn(print),
         ],
         stepDefinitions: [
           appLoad(),
           grantPermissionWhenInUse(),
           takePatrolScreenshot(),
         ],
         createWorld: (TestConfiguration config) =>
             worldCreator(config, nativeAutomator, hostAutomator),
       ),
       appMainFunction: (World world) => app.main(),
     );
   }

   My example steps that can access nativeAutomator and hostAutomator respectively.

   import "package:gherkin/gherkin.dart";
   
   import "../my_world.dart";
   
   StepDefinitionGeneric grantPermissionWhenInUse() {
     return given<MyWorld>(
       RegExp(r"I grant location permission when in use$"),
       (context) async {
         context.world.nativeAutomator.grantPermissionWhenInUse();
       },
     );
   }

   import "package:gherkin/gherkin.dart";
   
   import "../my_world.dart";
   
   StepDefinitionGeneric takePatrolScreenshot() {
     return given<MyWorld>(
       RegExp(r"I take screenshot$"),
       (context) async {
         context.world.hostAutomator.takeScreenshot();
       },
     );
   }

5. Use flutter pub run build_runner build to generate test (because of flutter_gherkin v3) but use patrol drive --driver=test_driver/integration_test.dart --target=integration_test/gherkin_suite_test.dart to run the test.

Done. Happy coding!

[bartekpacia]

This looks awesome! Thanks a lot for sharing this with us :)

[sandesh181199]

Hi @nonvachara-paloit , Is it still working for you? Currently I am using patrol 3.0.1 and I am facing issues with above solution. It will be really helpful if you could provide the repository of your implementation if possible.

I have completed the implementation with flutter gherkin but facing issues with Patrol integration for Native automation.

[nonvachara-paloit]

Hi @sandesh181199 .
Unfortunately, this snippet is not working anymore since it was from patrol 0.x.
And I no longer actively POC with Patrol.

I've seen some possible alternative. Please have a look at it here https://pub.dev/packages/gits_cucumber

[sandesh181199]

Hi @nonvachara-paloit ,
Thank you for the reply.
Yes, I was trying to change few things to make it work for Patrol 3.x but was not able to. Currently, I have a complete framework build using flutter_gherkin. Could you please suggest if I can migrate it to gits_cucumber and will it be helpful as gits_cucumber have comparatively less popularity and is very new. Please suggest