Skip to content

ci: implement GitHub Actions runs for android ui tests (WPB-23413)#4605

Open
emmaoke-w wants to merge 33 commits intodevelopfrom
ci/WPB-23413-qa-ui-tests-v1
Open

ci: implement GitHub Actions runs for android ui tests (WPB-23413)#4605
emmaoke-w wants to merge 33 commits intodevelopfrom
ci/WPB-23413-qa-ui-tests-v1

Conversation

@emmaoke-w
Copy link
Contributor

@emmaoke-w emmaoke-w commented Feb 19, 2026

https://wearezeta.atlassian.net/browse/WPB-23413


PR Submission Checklist for internal contributors

  • The PR Title

    • conforms to the style of semantic commits messages¹ supported in Wire's Github Workflow²
    • contains a reference JIRA issue number like SQPIT-764
    • answers the question: If merged, this PR will: ... ³
  • The PR Description

    • is free of optional paragraphs and you have filled the relevant parts to the best of your ability

What's new in this PR?

  1. Safe concurrency (device-aware locking)
  • Uses a concurrency group keyed by ${{ inputs.androidDeviceId || 'auto' }}.
  • If a specific device is requested, it locks that device lane.
  • If device id is empty (auto), it locks the shared auto-run lane to avoid pool collisions.
  1. Input validation + selector resolution
  • Validates upgrade inputs (oldBuildNumber required when isUpgrade=true).
  • Parses TAGS into either:
  • testCaseId (for @TC-xxxx)
  • category (for tags like @regression)
  • Rejects unsupported selector format @key:value with a clear error.
  1. Flavor resolution from runner config (single source of truth)
  • Resolves flavor from runner-local /etc/android-qa/flavors.json (not hardcoded in repo).
  • Exports S3_FOLDER, APP_ID, and PACKAGES_TO_UNINSTALL via $GITHUB_ENV.
  • Flavor input acts as a lookup key into runner config.
  1. APK retrieval from S3 (new + optional old)
  • Lists .apk keys under the resolved flavor prefix.
  • Resolves target APK using:
  • latest (highest parsed version)
  • direct APK filename/path input
  • build token substring match
  • For upgrade runs, resolves and downloads both old and new APKs.
  1. Device targeting + install behavior
  • Detects connected devices via adb devices.
  • Supports:
  • androidDeviceId set -> target that single device
  • empty device id + @TC-* mode -> target one device
  • empty device id + non-@TC-* mode -> target all connected devices
  • Uninstalls configured package ids before install (PACKAGES_TO_UNINSTALL).
  • Supports downgrade install when enforceAppInstall=true (adb install -d).
  1. Runtime secrets via 1Password
  • Installs 1Password CLI in workflow.
  • Fetches runtime secrets.json into ${RUNNER_TEMP} and symlinks into repo for test runtime.
  • Masks OP_SERVICE_ACCOUNT_TOKEN in logs.
  • Removes runtime secrets before reporting/publishing and in final cleanup.
  1. Test execution model (parallel instrumentation per device)
  • Builds test APK once, then runs instrumentation in parallel (one background worker per selected device).
  • Uses sharding args:
  • numShards = deviceCount (or 1 for single test case mode)
  • per-device shardIndex
  • Uses TaggedFilter with resolved testCaseId/category.
  • Treats both non-zero instrumentation exit and known failure markers in logs as failures.
  1. Allure reporting + GitHub Pages publishing
  • Pulls Allure results from each device.
  • Merges results and enriches metadata:
  • device label (model + android version + serial)
  • run/build/tags in environment.properties
  • executor.json linked to GitHub Actions run
  • Generates Allure HTML and publishes to gh-pages/docs/qa-ui-tests.
  • Applies retention cleanup (KEEP_DAYS=90).
  • Adds direct report link to GitHub Actions Step Summary.
  1. Retry block removed
  • No retry-on-fail execution block is included in this workflow version.
  1. Scripted workflow refactor for maintainability
  • Moves major inline workflow logic into dedicated scripts under scripts/qa_android_ui_tests/.
  • Workflow now orchestrates clear phase commands (validation, setup, execution, reporting).
  • Includes checkout in the validation job so script files are available on that runner.
  1. Hardening improvement in APK env handling
  • Replaced shell source of generated APK env file with strict key parsing and allowlisted keys.
  • Prevents accidental shell execution from malformed/untrusted generated content.

Issues

Briefly describe the issue you have solved or implemented with this pull request. If the PR contains multiple issues, use a bullet list.

Causes (Optional)

Briefly describe the causes behind the issues. This could be helpful to understand the adopted solutions behind some nasty bugs or complex issues.

Solutions

Briefly describe the solutions you have implemented for the issues explained above.

Dependencies (Optional)

If there are some other pull requests related to this one (e.g. new releases of frameworks), specify them here.

Needs releases with:

  • GitHub link to other pull request

Testing

Test Coverage (Optional)

  • I have added automated test to this contribution

How to Test

Trigger manually via Actions → “QA Android UI Tests” → Run workflow.

Typical examples:

Latest build / all devices

appBuildNumber: latest

androidDeviceId: (empty)

Single device

-androidDeviceId:

-Single testcase

-TAGS: @TC-8143

-Category run

-TAGS: @regression

-Upgrade run

-isUpgrade: true

-appBuildNumber: latest

-oldBuildNumber:

Briefly describe how this change was tested and if applicable the exact steps taken to verify that it works as expected.

Notes (Optional)

Specify here any other facts that you think are important for this issue.

Attachments (Optional)

image

Attachments like images, videos, etc. (drag and drop in the text box)


PR Post Submission Checklist for internal contributors (Optional)

  • Wire's Github Workflow has automatically linked the PR to a JIRA issue

PR Post Merge Checklist for internal contributors

  • If any soft of configuration variable was introduced by this PR, it has been added to the relevant documents and the CI jobs have been updated.

References
  1. https://sparkbox.com/foundry/semantic_commit_messages
  2. https://github.com/wireapp/.github#usage
  3. E.g. feat(conversation-list): Sort conversations by most emojis in the title #SQPIT-764.

@pull-request-size
Copy link

Ups 🫰🟨

This PR is too big. Please try to break it up into smaller PRs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing we need to do, for clarity and maintainability, is to have all the scripts in here in their own file rather than writing it on this workflow so we only call them here later.

We have the folder scripts/ to put them, python, bash you name it, let's move it there for clarity.

Take a look for example at https://github.com/wireapp/wire-android/blob/develop/scripts/generate_whatsnew_notes.py

@sonarqubecloud
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants