Test automation is not a common phrase in the game industry, especially when compared to other tech industries. While there has been progress in the tooling made available to developers, and Epic has some documentation on how to use functional testing in Unreal Engine, it’s a bit lackluster and overall there remains a lack of detailed information. In particular, it’s hard to find examples for others to learn from so they can really utilize the framework.

Today I’ll help bridge that gap by sharing how to get started with the Functional Test framework using the default First Person template in Unreal 5.4.

Note: I have written this to be accessible to people relatively new to Unreal, but requires basic knowledge of how to use Blueprints (navigating the engine, movement on levels, adding actors to levels, creating files, creating nodes and connecting them).

What is functional testing?

Functional testing in the realm of software testing is essentially making sure a feature works as intended. For example, when examining a combat system, functional tests might verify that damage output from different weapons is accurate, armor mitigates damage as expected, and attacks can be dodged. In order to test this manually, you’d have to spin up the engine (or compiled game), open a level and initiate the actions yourself, validating its functionality with your eyes and whatever logs or resources made available to you.

Unreal’s functional testing framework

The functional testing framework is designed to test these kinds of features and save the time it takes to run these tests manually — in particular, saving the time it takes to run these tests repeatedly over the development of a game. These tests can be created in Blueprints or C++.
High level, creating tests in the functional framework looks like the following:

1. Activate the plugin
2. Create a testing level containing any necessary actors in which to run the test, including a test actor
3. Create a test actor and write a test. Repeat as needed.
4. Run the tests

For anyone familiar with writing tests in code, Epic’s approach makes the Level like the test class and Functional Test actors the test functions. As far as I’ve found, there is no way to have more than one test per test actor or more than one assertion per test actor. It’s an adjustment, but it does force abstraction that you can typically skip over when writing automation for websites or mobile apps.

Activate the functional testing plugin

In order to use the framework, you need to activate the Functional Testing plugin.

1. Go to Edit -> Plugins
2. Go to the Testing filter or search for “functional”
3. Activate the Functional Testing Editor plugin
4. Restart Unreal to use the plugin

Set up the Player Character

For the sake of example, I’ll start off with some tests for stats on the Player Character. These specific tests aren’t an ideal case for Functional tests, but are easy to get up and going. The reason is that each individual test takes between 0.3 and 0.5 seconds to run each. Unit tests are much better for these particular kinds of tests. 

The setup:

1. Added a float of CurrentHealth. Default to 1.0.
2. Added a boolean of isAlive. Default to true.
3. Added a function called TakeDamage. This function should take in a float damageAmount, subtract it from CurrentHealth and check if it is less than or equal to 0. Connect to a branch node. On true set the health to zero and set isAlive to false. On false, set the CurrentHealth to the result of CurrentHealth–DamageHealth.

Create a test level

In order to detect and run tests, they need to be associated with a test level. 

1. Go to File -> New Level
   2. Select the basic level
   3. Add any actors needed for the test (such as the Player Character, a button toggle, enemies, etc)

For a basic test (again, for the sake of example and not the type of tests I’d recommend for functional testing), I decided to go with testing player health and damage. Since we need the player character for this, I added it to the level.

This is all we’ll need on the level for now, so now its time to set up the first test.

Creating a basic test

There are two places to create functional tests — in the Level blueprint itself, or in a Functional Test blueprint actor. In terms of execution, it doesn’t seem like there’s much difference, so where you choose is more about your preferences for organization and how you set the tests up.
There are a few default nodes used for tests. I’ll discuss the primary ones below, but you can see the others in Unreal’s docs under Functional Test Class Features and their Functional Testing API Reference. 

• Prepare Test. This is where any code for setting up the tests is placed. This can be spawning actors, streaming level information, or something like setting references to variables for the test itself.
• On Test Start. Where the test execution code goes. 
• On Test Finished. A delegate where any cleanup code goes. Remove anything spawned in, reset any modified data, etc to ensure that you don’t break subsequent tests.

Creating tests in the Level

For the test in the level, we’ll validate that the TakeDamage function updates the player health to the correct value by making a simple test that checks if the player is damaged for a quarter of their health, 3/4 of it will remain.

1. In the level editor, go to the Place Actors section and search for Functional Test. Drag it into the level. Name it something meaningful, as this name will be displayed in the test runner. I named mine FT_Level_PlayerCanTakeDamage.

2. Select the functional test actor in the level, then open the level blueprint and create a On Test Start node. It should show the name of the functional test you dragged in.
3. Set up the test code. 

Since we’re looking at ensuring the TakeDamage function updates the player health stat correctly, we’ll need to reference BP_FirstPersonCharacter to access it’s variables.

Creating the test code

Create variables

• PlayerCharacterof type BP_FirstPersonCharacter
• TestActorof type FunctionalTest

Set up the Test Prepare

• Set references to the Blueprint Actor and Test Actor to be used in the test. To create the reference for the set, click the actor in the level, then go to the blueprint and right click -> create reference.

Set up the test

1. Drag in the player and call the TakeDamage function. Pass in a damage amount of 0.25.
2. From PlayerCharacter, drag the pin to create a CurrentHealth variable reference. Connect that to a == node. Set the comparison value to 0.75.
3. Create a branch node and connect to the equality result.
4. In the branch, drag true and create a Finish Test node, and set the Result to true. Do the same for the False path and select the result to false. Set an informative message on failure.

Creating tests in a blueprint actor

For the blueprint actor test, we’ll ensure that if the player health is 0 their isAlive state is set to false.

1. Create a new Functional Test blueprint actor. Right Click -> Blueprints search for Functional Test. Create the blueprint. I named mine BPT_PlayerIsAlive. Unlike with the Level tests, this name will not show in the test runner.
2. Add variables for our necessary references and assertion values. In this case we’ll want PlayerCharacterof type BP_FirstPersonCharacter, shouldPlayerBeAliveof type Boolean, and Damageof type Float. Set all these values to public by clicking on the eye next to them — we’ll need to modify these later in the level blueprint.

3. Create the Start Test node and add the test code. Call the TakeDamage function, pass in the Damage variable. Then do a check to see if the isAlive flag on the player character matches the expected shouldPlayerBeAlive boolean. Pass or fail the test based on the results. Save and compile.

Setup of the test

4. Add the blueprint to the TestLevel and set up the variables. Drag in the function then select it in the level and view the details panel. There should be a Default section that shows the three variables we created. In the dropdown for PlayerCharacter, select BP_FirstPersonCharacter, set damage to 1.0, and make sure shouldPlayerBeAlive is false.

(Optional) Additional Settings for both types of tests

There are two additional sections of note that are accessible to both test types when viewing the details panel for the test actor in the level.

Functional Test Section

This section has a few additional options for tests.

• Owner and description are just informative, giving you the option to say who wrote/maintains the test and a brief description about it if it’s not obvious. 
• Is Enabled, as the name implies, is a toggle to enable or disable the test.
• Log Error Handling just lets you specify how any output is handled — engine default, as an error, or ignored.
• Log Warning Handling is the same as error handling, but for warnings.
• Observation Point is interesting — by default, when tests are run a window opens so you can observe what is going on. It will default to the PlayerCharacter’s camera if you have one. If you add another camera into the level you can select it and set it as the Observation Point. The test will then be viewed from that angle. You can also move around and observe a test in progress as a spectator.

Timeout Section

This section allows you to set up your default timeouts. I recommend setting the time limit to something small — for the tests written in this post, I have it set to 5 seconds.

Preparation Time Limit is how long the test will wait for a Ready flag if you’re using the IsReady node in the Prepare Test event. This is used to make sure a condition is met before you run the test, although I haven’t sorted how to work with it yet myself.

Time Limit is how long the test has to run before it hits a timeout failure. Keep this number as small as you can to keep run times down.

Times Up Message is just the message that displays when the test fails due to timeout.

Times Up Result has the same options as the End Test Node — Default, Invalid, Error, Running, Failed, Succeeded.

Running tests in the session runner

When running tests in-editor, you can use the session runner.

To open it, go to Tools -> Session Frontend. Note that this option will only be visible when the Functional Testing Framework plugin is activated and Unreal has been restarted, as mentioned at the beginning of this post.
Under the Automation tab, there will be a long list of tests that exist in the engine already. To see our tests, go to Project -> Functional Tests -> Tests -> {nameOfTestLevel}.

To run a test, just check the box next to it and hit the play button at the top. You’ll see a window pop up with the view of the First Person camera by default, and then it’ll close when the tests completes.

If you select both tests created, they will run sequentially.

I haven’t gone too deep into the session frontend yet — that’s for future investigations. But that’s the basics of running a test in the editor.

Running tests in command line

Tests can also be run in command line. In my experience so far, this has been very slow as it defaults to opening a new instance of Unreal. I by far recommend running from the Session Frontend during development executions of the tests. There’s probably ways to improve runtime performance, but I haven’t figured it out yet.

Shout out to Doug with his 2016 answer on StackOverflow that helped out with this section.

The command boils down to the following components:

1. Launch the engine. /{UnrealEditorLocation}
2. Open the project at a given location /{ProjectLocation}/{ProjectName}.uproject
3. Specify the tests to run. You can specify all tests, all functional tests, or just tests associated with a specific test level ExecCmds="Automation RunTests project.functional tests.tests.{OptionalNameOfTestLevel}"
4. Optionally, specify the location of the output report. When the tests are done running, they’ll generate a JSON object with the results that can be used in CI. -ReportOutputPath="{OutputLocation}"

Putting it together

Mac

/Users/Shared/Epic\ Games/UE_5.4/Engine/Binaries/Mac/UnrealEditor.app/Contents/MacOS/UnrealEditor /{path}/{projectName}.uproject -ExecCmds="Automation RunTests project.functional tests.tests" -ReportOutputPath="{outputLocation}"

Windows

.\UnrealEditor.exe "{PathToProject}/{ProjectName}.uproject" -ExecCmds="Automation RunTests project.functional tests.test.{OptionalNameOfTestLevel}" -ReportOutputPath="{OutputLocation}"

Check out this project on github to see these examples and more. Stay tuned for more explorations into Unreal automation. 

Have you experimented with Functional testing in Unreal? Have any clarifications or corrections? If so, comment and share your best tips!