# Android end-to-end test automation

Recapping: In the [A journey through end-to-end tests](https://blog.biped.works/smoke-test) article (Please, read it first) we was introduced to the `end-to-end`, `user journey test` or `smoke tests` terms. We also saw how to write test scenarios using Gherkin Syntax to structure our Domain Specific Language and here we are...

Looking to the following scenario, how can we automate it?  
`Given my application is installed`  
`And I tap on the app icon`  
`When the app is launched`  
`Then I see the Login screen`

### Rules

By observing the given test scenario It's possible to extract some rules and infer some others.

* Test scenarios are composed by ordered steps
    
* Each step should be asserted  
    It's possible to find the exact step that caused a failure.
    
* Each step execution result should be printed  
    As each step should be asserted, each step execution result should also be printed as It is described in the test scenario (e.g. And I tap on the app icon FAILED).
    
* Test scenarios should also be ordered  
    As we're simulating the real world, it should not be possible to open the account screen before authenticate an user (If the application requires authentication, of course)
    

Now that we have the test rules, let's select the tools that we're going to use to automate our tests.

### Tools

On android we have two kind of instrumentation tools that helps to simulate the user interactions, `Espresso` and `UiAutomator`.  
With `espresso` we have a sort of gray test box, as it allows the developer to access and modify some application values, on the other hand the UiAutomator simulate a real black box, where we interact with the app and the system (not possible on espresso) as a real user, and that's why we'll choose the UiAutomator.

<div data-node-type="callout">
<div data-node-type="callout-emoji">❗</div>
<div data-node-type="callout-text">We'll not cover the UiAutomator details in this text, please take a look in the docs <a target="_blank" rel="noopener noreferrer nofollow" href="https://produto.mercadolivre.com.br/MLB-3366097983-tampo-mesa-madeira-macica-setup-escritorio-balco-200x80cm-_JM?matt_tool=18983347&amp;matt_word=&amp;matt_source=google&amp;matt_campaign_id=14302215531&amp;matt_ad_group_id=155854918005&amp;matt_match_type=&amp;matt_network=g&amp;matt_device=c&amp;matt_creative=686778909993&amp;matt_keyword=&amp;matt_ad_position=&amp;matt_ad_type=pla&amp;matt_merchant_id=235933234&amp;matt_product_id=MLB3366097983&amp;matt_product_partition_id=1963044459753&amp;matt_target_id=aud-1966009190540:pla-1963044459753&amp;cq_src=google_ads&amp;cq_cmp=14302215531&amp;cq_net=g&amp;cq_plt=gp&amp;cq_med=pla&amp;gad_source=1&amp;gclid=CjwKCAjwrvyxBhAbEiwAEg_Kgow6eW3-mnAEJRMkT-Yn4YbL2ylNj1gJ7g-E88UlzRdtv6xgONnpjxoCEvMQAvD_BwE#reviews" style="pointer-events: none">here</a>.</div>
</div>

Ok, what about the Junit? Is it possible to use Junit5? May it have some new feature that can help? Yes, it does, but the Junit5 is not officially supported on android (They also don't have plans for that) and the setup is not simple for the instrumentation tests. Let's keep using the Junit4 instead of adding a bunch of unnoficial library to our project.

And what about cucumber? You can give it a try, but It's not easy to configure, especially with UiAutomator. We can have something simpler by creating a custom test runner with the Junit4, let me show you how.

### Test Runner

We have three important rules set for our test steps, they should be ordered, asserted and properly logged, but Junit4 doesn't offer the `@Order` and `@DisplayName` annotations as Junit5, so, how we can comply with these step rules?

On Junit4 we have the possibility to extend `BlockJUnit4ClassRunner` and override the `getChildren` and `describeChild` methods, let me explain...

The test methods in the Junit4 are filtered/found by the `@Test` annotation inside the `Junit Test Runner`, this process happens in the `getChildren` method, and then each method found is executed by calling the `runChild` (Undefined execution order), which also calls the `describeChild` method to define the test name to be displayed. Now we have all the information needed to modify the `Junit Runner` as we want.

So, the `@Test` annotation is just an information to Junit, and the same is valid for the `@Ignore` annotation, that tells Junit to just logging the given method as ignored instead of running it. With that, we can also add more information to the test method, as telling Junit the sort `order` of a test for example.

So let's create an annotation called `@Order` and try it.

```kotlin
@Retention(AnnotationRetention.RUNTIME)
@Target(AnnotationTarget.CLASS, AnnotationTarget.FUNCTION)
annotation class Order(@IntRange(from = 0L, to = 100) val value: Int = 0)
```

Now it's time to create our custom test runner extending `BlockJUnit4ClassRunner`

```kotlin
class AutomatorRunner(
    private val testClass: Class<*>
) : BlockJUnit4ClassRunner(testClass)
```

And then we can sort the test methods by overriding the `getChildren` method for that

```kotlin
 override fun getChildren(): MutableList<FrameworkMethod> {
    return sortTestClassMethods()
         .map { FrameworkMethod(it) }
         .toMutableList()
}
 
private fun sortTestClassMethods(): List<Method> {
    return testClass.methods
        .filter { it.getAnnotation(Test::class.java) != null }
        .sortedBy { it.getAnnotation(Order::class.java)?.value ?: 0 }
}
```

In the code above we first filter the existent test (annotated with `Test::class`) methods inside the `Test Class` sorting them using the `Order::class` annotation value.

Now let's create our test class using our the `AutomatorRunner` along with `Order` annotation.

```kotlin
@RunWith(AutomatorRunner::class)
class ApplicationStartScenario {
    @Test
    @Order(1)
    fun givenMyApplicationInstalled() ...
        
    @Test
    @Order(2)
    fun andITapOnTheIcon() ...
    
    @Test
    @Order(3)
    fun whenTheApplicationIsOpened() ...
    
    @Test
    @Order(4)
    fun thenISeeTheLoginScreen() ...
}
```

Running the test above will give us the following output:

`givenMyApplicationInstalled SUCCESS`  
`andITapOnTheIcon SUCCESS`  
`whenTheApplicationIsOpened SUCCESS`  
`thenISeeTheLoginScreen SUCCESS`

Not that readable, right? Let's fix it. We need a display name and we have the option to create a `DisplayName` annotation or enhance the `Order` annotation, which is the selected approach here.

The `@Order` annotation can be renamed to `@Step`, as our test `Scenarios` are composed by `Steps`, then we just need to add the `displayName: String` field to the `Step::class`

```kotlin
@Retention(AnnotationRetention.RUNTIME)
@Target(AnnotationTarget.FUNCTION)
annotation class Step(
    val displayName: String,
    @IntRange(from = 0L, to = 100) val order: Int = 0
)
```

To modify the test method display name we need to override the `describeChild` method

```kotlin
override fun describeChild(frameworkMethod: FrameworkMethod): Description {
    val displayName = getDisplayName(frameworkMethod.method)

    return Description.createTestDescription(
        testClass.name,
        displayName,
        frameworkMethod.method.annotations
    )
}
    
private fun getDisplayName(method: Method): String {
    val annotation = method.getAnnotation(Step::class.java)
    return annotation?.displayName ?: method.name
}
```

That's it, if the method don't have the `@Step` annotation we return the test method name, if it's annotated, then we return the `displayName` value from `@Step`.

Here's the final version of our Automator Class

```kotlin
class AutomatorRunner(private val testClass: Class<*>) : BlockJUnit4ClassRunner(testClass) {

    override fun getChildren(): MutableList<FrameworkMethod> {
        return sortTestClassMethods().map { FrameworkMethod(it) }.toMutableList()
    }

    override fun describeChild(frameworkMethod: FrameworkMethod): Description {
        val displayName = getDisplayName(frameworkMethod.method)

        return Description.createTestDescription(
            testClass.name,
            displayName,
            frameworkMethod.method.annotations
        )
    }

    private fun sortTestClassMethods(): List<Method> {
        return testClass.methods
            .filter { it.getAnnotation(Test::class.java) != null }
            .sortedBy { it.getAnnotation(Step::class.java)?.order ?: 0 }
    }

    private fun getDisplayName(method: Method): String {
        fun formatMethodName(name: String): String {
            return name.replace("_", " ")
        }

        val annotation = method.getAnnotation(Step::class.java)
        return annotation?.displayName ?: formatMethodName(method.name)
    }
}
```

Time to update our `ApplicationStartScenario` test class

```kotlin
@RunWith(AutomatorRunner::class)
class ApplicationStartScenario {
    @Test
    @Step("Given My Application Installed", 1)
    fun givenMyApplicationInstalled() ...
        
    @Test
    @Step("And I tap on the app icon", 2)
    fun andITapOnTheIcon() ...
    
    @Test
    @Step("When the app is opened", 3)
    fun whenTheApplicationIsOpened() ...
    
    @Test
    @Step("Then I see the Login Screen", 4)
    fun thenISeeTheLoginScreen() ...
}
```

The test example above would print:  
`Given My Application Installed SUCCESS`  
`And I tap on the app icon SUCCESS`  
`When the app is opened SUCCESS`  
`Then I see the Login Screen SUCCESS`

Cool, but how we can have `Ordered Scenarios`? Test Suite

### JUnit Test Suite

Ordered scenarios means that we should call the test classes in a sort order, and Junit4 already give us this feature, the `Test Suite`, here's how to use it..

```kotlin
@RunWith(Suite::class)
@Suite.SuiteClasses(
    ApplicationStartScenario::class,
    LoginScenario::class,
    HomeScenario::class
)
class SmokeTestSuite
```

That's it, the class order that goes inside @Suite.SuiteClasses represents that execution order of the `SmokeTestSuite`.