What are XCUI test frameworks

AWS Device Farm FAQs

Test the app

Q: I don't have any automated test scripts yet. How do the preconfigured tests work?

With the preconfigured test suite for testing compatibility, you can install, uninstall, start and run Fuzz on the app.

Q: What is the function of Fuzz?

Fuzz will perform fuzzing on your user interface right after starting. Fuzz streams the fast, random user input (typing, swiping, keyboard input) into your app. You can configure the number of events, the interval between events, and the starting value to randomize the events. If you use the same seed for multiple test runs, the events will run in the same order.

Q: I'm testing an automation framework. Which frameworks do you support?

For testing iOS, Android and FireOS apps, we currently support Appium Java JUnit, Appium Java TestNG, Appium Python, Calabash, Instrumentation (including JUnit, Espresso, Robotium and any instrumentation-based tests), UI Automation, UI Automator and XCTest ( including XCUI and KIF). Further information and a current list can be found in our documentation.

Q: What test frameworks are supported for web applications?

You can run tests written in Appium Java JUnit, Appium Java TestNG, or Appium Python.

Q: Can you add support for modified or custom frameworks?

We always try to support new frameworks. Please contact us.

Q: How does AWS Device Farm decide when to take a screenshot during the test?

If you use one of the supported automation frameworks, you have complete control and control over when screenshots are taken. These screenshots are automatically included in your reports.

Q: Android: Is Google Play Services available on your devices? Which version is installed?

Yes. Google Play Services is installed on the devices that support the service. The services are updated with new versions.

Q: Android: Do the devices have a standard Google account?

No. There is no active Google account on the devices.

Q: Does AWS Device Farm support automation recording and playback, or do I have to write my own scripts?

AWS Device Farm supports frameworks such as Espresso and Robotium, which include scripting tools for recording and playback.

Q: iOS: Do I need to add your UDIDs to my provisioning profile?

No, AWS Device Farm automatically replaces a provisioning profile and abandons your app so it can be provisioned on your devices.

Q: iOS: My app does not contain debugging symbols. Can I equip AWS Device Farm with a dSYM file?

No, but you can download the log files and symbolize the stack traces locally.

Q: Android: My app is using obfuscation. Can I still test my app on AWS Device Farm?

Yes, if you are using ProGuard. If you use DexGuard with the anti-piracy features, we will not be able to re-sign the app and run the tests again.

Q: My app is showing ads. Is the advertisement showing on your device? Will my advertiser classify this as abuse and block my account?

Although the devices have internet access, we cannot guarantee that the advertisements will be displayed. We recommend removing the ads before building tests on AWS Device Farm.

Q: Do I have access to the machine hosting my device or the shell as part of my testing? Can I use it to connect to the Internet?

Yes. If you are using a client-server framework like Calabash, Appium, or UI Automation, you can access the internet and run limited shell commands from the host.

Q: I want to send media or other data for use in my app. How do I do this?

You can send a ZIP archive up to 4 GB. In Android it is extracted in the root directory of the external storage, in iOS in the sandbox of your app. Android extension files (OBB) are automatically moved to the appropriate location for the operating system. Refer to the developer's guide for more information.

Q: My app needs dependencies to test all functions. Can i install other apps?

Yes. You can choose multiple apps and also the order in which they are installed. The dependent apps are then installed before the tests start.

Q: Can I test upgrade processes for my app? How do I install an older version of my app?

Yes. To test your upgrade process, you can upload and install an older version of your app before the new version is installed and tested.

Q: My app uses location services. Can I provide the physical location of the device?

Yes. You can specify the appropriate coordinates for longitude and latitude that will be used to overwrite a device's GPS.

Q: Can I run localization tests? How do I change the device language?

Yes. You can provide a language (e.g. "de_DE") to overwrite the default language setting on the device.

Q: How long will it take for my test to start?

Tests are immediately queued for execution and typically start within minutes. If one or more devices are not available, the test execution for those devices will remain in the queue until the devices are available. The tests on other devices are carried out in the meantime.

Q: What is the maximum test duration?

The maximum time allowed is 60 minutes. If you need a longer execution time, please contact us.

Q: Does AWS Device Farm provide the ability to run tests through an API and download the results through it?

Yes. We have a plugin for the Jenkins environment for ongoing integration and a Gradle plugin that is compatible with Android Studio. AWS Device Farm also provides programmatic support for all console functions including setting up a test and downloading test results via an API. For more information, see the AWS Device Farm API Guide. In addition to the API, you can also access AWS Device Farm through the AWS SDKs.