Spectating System in Unreal Engine

Ever wondered how you can make a Spectating System and then immediately panicked? Well guess you don’t have to panic anymore, as it’s easier than you thought! While in theory there could be several ways to implement this system, I’m going to walk you through the engine way of doing it!

Introduction

The Spectating System lets some type of players spectate some other type of players. Spectators are usually dead players, and those Spectated are alive players. While it has many benefits, the most prominent is that it makes dead players view the game flow so they are busy with something till they respawn.

Spectator Pawn

The SpectatorPawn class is a subclass of DefaultPawn. Through a GameMode, different classes can be specified as defaults for Pawn and SpectatorPawn, and this class provides a simple framework ideal for spectating functionality. We will use it mainly to setup the spectating input.

The header file of our custom subclass of SpectatorPawn class will have the following function declarations:

MySpectatorPawn.h

virtual void SetupPlayerInputComponent(UInputComponent* InInputComponent) override;

/** Move camera to next player */
void ViewNextPlayer();

/** Move camera to previous player */
void ViewPrevPlayer();

The source file will have their implementations:

MySpectatorPawn.cpp

void AMySpectatorPawn::SetupPlayerInputComponent(UInputComponent* InInputComponent)
{
    Super::SetupPlayerInputComponent(InInputComponent);

    InInputComponent->BindAction("ViewNext", IE_Pressed, this, &ThisClass::ViewNextPlayer);
    InInputComponent->BindAction("ViewPrev", IE_Pressed, this, &ThisClass::ViewPrevPlayer);
}

void AMySpectatorPawn::ViewNextPlayer()
{
    if (APlayerController* PC = GetController<APlayerController>())
    {
        PC->ServerViewNextPlayer();
    }
}

void AMySpectatorPawn::ViewPrevPlayer()
{
    if (APlayerController* PC = GetController<APlayerController>())
    {
        PC->ServerViewPrevPlayer();
    }
}

Note that both ServerViewNextPlayer and ServerViewPrevPlayer are part of the source code, and they are two core functions of the built in Spectating System.
The ActionMappings ViewNext and ViewPrev were configured using the old Input System (the new one is called Enhanced Input System).

Where does This Class Exist?

To answer this question we have to look at how the SpectatorPawn is spawned, and that is done via APlayerController::SpawnSpectatorPawn(). We can easily tell that it exists only at the local PlayerController level, i.e server-side for the listen-server player in case of a listen-server setup, and client-side for each other player in either one of the setups (listen or dedicated server), and that it’s possessed by the local PlayerController.

PlayerController.cpp

ASpectatorPawn* APlayerController::SpawnSpectatorPawn()
{
    ASpectatorPawn* SpawnedSpectator = nullptr;

    // Only spawned for the local player
    if ((GetSpectatorPawn() == nullptr) && IsLocalController())
    {
        ...

        SpawnedSpectator = World->SpawnActor<ASpectatorPawn>(SpectatorClass, GetSpawnLocation(), GetControlRotation(), SpawnParams);
        if (SpawnedSpectator)
        {
            SpawnedSpectator->SetReplicates(false);
            SpawnedSpectator->PossessedBy(this);
        }

        ...
    }

    return SpawnedSpectator;
}

This means that we can’t replicate properties or route meaningful RPCs through the SpectatorPawn class. Though considering the fact that it’s possessed by the PlayerController class, we can retrieve and use the latter for such tasks as we’ve already seen it above in ViewNextPlayer() & ViewPrevPlayer() class methods.

Note: Originally, this what was written: SpawnedSpectator->SetReplicates(false); // Client-side only. I omitted the comment as it was totally misleading!
As we said above, that’s not the case for a listen-server player, in which his SpectatorPawn is server-side only.

Creating The HUD Infrastructure

As of nowadays terms this class is considered a legacy, though it’s still a perfect place to manage all UMG widgets.
It only exists on client, so it’s not suitable for all kinds of replication. It is owned and easily accessible by the PlayerController.
We will use it to define an EHUDState enum class to make the HUD display different UMG widgets depending on the player’s gameplay state.

MyHUD.h

UENUM(BlueprintType)
enum class EHUDState : uint8
{
    Playing,
    Spectating,
    Inactive
};

UCLASS()
class AMyAwesomeActor : public AActor
{
    GENERATED_BODY()

/* Current HUD state */
EHUDState CurrentState;

public:
UFUNCTION(BlueprintCallable, Category = "HUD")
EHUDState GetCurrentState() const;

/* Event hook to update HUD state (eg. to determine visibility of widgets) */
UFUNCTION(BlueprintNativeEvent, Category = "HUDEvents")
void OnStateChanged(EHUDState NewState);
}

MyHUD.cpp

EHUDState AMyHUD::GetCurrentState() const
{
    return CurrentState;
}

void AMyHUD::OnStateChanged_Implementation(EHUDState NewState)
{
    CurrentState = NewState;
}

System Core

Most, if not all of the spectating functionality can be found in APlayerController class.
Let’s subclass it first, then declare, override and define our methods accordingly:

MyPlayerController.h

/** Set Player to play. Should be called only on server */
UFUNCTION(BlueprintCallable, BlueprintAuthorityOnly)
void SetPlayerPlay();

/** Set Player to spectate. Should be called only on server */
UFUNCTION(BlueprintCallable, BlueprintAuthorityOnly)
void SetPlayerSpectate();

/** Notify HUD of a state change so it shows suitable widgets accordingly */
UFUNCTION(Reliable, Client)
void ClientHUDStateChanged(EHUDState NewState);

MyPlayerController.cpp

void AMyPlayerController::SetPlayerPlay()
{
    // Only proceed if we're on the server
    if (!HasAuthority())
    {
        return;
    }

    // Update the state on server
    PlayerState->SetIsSpectator(false);
    ChangeState(NAME_Playing);

    bPlayerIsWaiting = false;

    // Push the state update to the client
    ClientGotoState(NAME_Playing);

    // Update the HUD to remove the spectator screen
    ClientHUDStateChanged(EHUDState::Playing);
}

void AMyPlayerController::SetPlayerSpectate()
{
    // Only proceed if we're on the server
    if (!HasAuthority())
    {
        return;
    }

    // Update the state on server
    PlayerState->SetIsSpectator(true);
    ChangeState(NAME_Spectating);

    bPlayerIsWaiting = true;

    // Push the state update to the client
    ClientGotoState(NAME_Spectating);

    // Update the HUD to add the spectator screen
    ClientHUDStateChanged(EHUDState::Spectating);
}

void AMyPlayerController::ClientHUDStateChanged_Implementation(EHUDState NewState)
{
    if (AMyHUD* HUD = GetHUD<AMyHUD>())
    {
        HUD->OnStateChanged(NewState);
    }
}

As can be seen above, SetPlayerSpectate is the instigator function for spectating. While there are many useful stuff in there, I want to focus on the interesting APlayerController::ChangeState() & APlayerController::ClientGotoState().

Gameplay States

If we take a look at UnrealNames.inl file, we find all the hardcoded names that Unreal defines. Part of the hardcoded names defined there are the following:

UnrealNames.inl

// Optimized replication.
REGISTER_NAME(320,Playing)
REGISTER_NAME(322,Spectating)
REGISTER_NAME(325,Inactive)

Those are gameplay states that are meant to be set into the following property:

Controller.h

/** Current gameplay state this controller is in */
UPROPERTY()
FName StateName;

As can be seen, this property isn’t replicated, so we need to set it separately on server and client version of the controller. Luckily, that can be done with the handy methods ChangeState() & ClientGotoState().

Note: All state names need to be defined in name table in UnrealNames.h to be correctly replicated (so they are mapped to the same thing on client and server).

How does SpectatorPawn Spawn After All?

Whenever ChangeState(NAME_Spectating) is called on server, it calls APlayerController::BeginSpectatingState() which makes the PlayerController unpossess (server only) the player pawn. When ClientGotoState(NAME_Spectating) is called on server, ChangeState(NAME_Spectating) is called on client, which again calls APlayerController::BeginSpectatingState() which destroys any previously spawned SpectatorPawn, spawns a new one, and set it as the current one in use.

Finally I Can Spectate?

Congrats if you’ve made it this far, cause you finally have a working basic Spectating System:

However, like every other system out there, it has a handful of bugs waiting to be fixed.

Fixing Bugs

1. Clients Can’t Spectate

Some of you might notice this weird behavior where the client isn’t able to spectate. If you’re playing as a listen-server then you’ll notice that the server player, aka the host, is able though!
Ensure APlayerCameraManager.bClientSimulatingViewTarget is set to false (it’s defaulted to false), otherwise clients are handling setting their own ViewTarget and the server should not replicate it. We definitely don’t want such behavior, so let’s set this flag to false. To enforce this, set it in your custom PlayerCameraManager’s class constructor, and make sure it’s set to false in the subclassed BP PlayerCameraManager class in case such one exists.

2. SpectatorPawn not Spawning at Death Location

While it’s up to you to decide when spectating takes place (usually on player pawn death), you’ll notice that the SpectatorPawn is spawned at the player pawn APlayerController.SpawnLocation:

PlayerController.h

/** The pawn used when spectating (nullptr if not spectating). */
UPROPERTY()
ASpectatorPawn* SpectatorPawn;

/** The location used internally when there is no pawn or spectator, to know where to spawn the spectator or focus the camera on death. */
UPROPERTY(Replicated)
FVector SpawnLocation;

The default behavior can be seen in the method APlayerController::SpawnSpectatorPawn() at the following line:

PlayerController.cpp

SpawnedSpectator = World->SpawnActor<ASpectatorPawn>(SpectatorClass, GetSpawnLocation(), GetControlRotation(), SpawnParams);

Instead of spawning the SpectatorPawn at SpawnLocation, which is the spawn location of our pawn, we want to spawn it at PlayerCameraManager->GetCameraLocation(), which is the current camera location, or in the other words the player pawn death location. Before I discuss how I fix the issue, let’s look at the this handy function first:

PlayerController.h

/** Set the SpawnLocation for use when changing states or when there is no pawn or spectator. */
virtual void SetSpawnLocation(const FVector& NewLocation);

PlayerController.cpp

void APlayerController::SetSpawnLocation(const FVector& NewLocation)
{
    SpawnLocation = NewLocation;
    LastSpectatorSyncLocation = NewLocation;
}

As its documentation suggests, I will be calling the function above as we enter NAME_Spectating state:

MyPlayerController.cpp

void AMyPlayerController::BeginSpectatingState()
{
    if(PlayerCameraManager)
    {
        SetSpawnLocation(PlayerCameraManager->GetCameraLocation());
    }

    Super::BeginSpectatingState();
}

While some might override APlayerController::SpawnSpectatorPawn(), I chose to override APlayerController:BeginSpectatingState(), so the death location is always stored in the properties it was meant to be stored, making the system persistent, and also I so don’t clutter my custom PlayerController class.

3. Spectated Character Movement Looks Jittery

While this issue might not be noticeable in simple movement modes, i.e., walking or running, it will be a real hassle for other more complex movement modes. Before we discuss the solution I found, thanks to KaosSpectrum, let’s discuss the reasoning behind the problem.
Problem is CameraComponent (or, more specifically SpringArmComponent) is attached to the CapsuleComponent inside your custom Character class. The way CharacterMovementComponent works is, the capsule is always teleported, which isn’t smooth, and that’s why you see a jump every time the server sends a new capsule location in. The skeletal mesh is what actually is smoothed, and interpolated client-side. Thus, CameraComponent (or, more specifically SpringArmComponent) needs to be attached to the SkeletalMeshComponent instead. Yep, as simple as that!

4. Simulated Proxies Invisible to Spectator due to Net Relevancy

While you might not experience this bug at first glance, due to testing on small scale maps, you’ll notice it quite fast when testing on larger scale maps. Simulated proxies won’t render on spectator’s screen as the simulated proxy actor isn’t relevant for the SpectatorPawn. For example, if the simulated proxy is a weapon actor, then the corresponding function that decides what’s relevant and what’s not for such a generic actor is the following:

ActorReplication.cpp

bool AActor::IsNetRelevantFor(const AActor* RealViewer, const AActor* ViewTarget, const FVector& SrcLocation) const
{
    if (bAlwaysRelevant || IsOwnedBy(ViewTarget) || IsOwnedBy(RealViewer) || this == ViewTarget || ViewTarget == GetInstigator())
    {
        return true;
    }
    else if (bNetUseOwnerRelevancy && Owner)
    {
        return Owner->IsNetRelevantFor(RealViewer, ViewTarget, SrcLocation);
    }
    else if (bOnlyRelevantToOwner)
    {
        return false;
    }
    else if (RootComponent && RootComponent->GetAttachParent() && RootComponent->GetAttachParent()->GetOwner() && (Cast<USkeletalMeshComponent>(RootComponent->GetAttachParent()) || (RootComponent->GetAttachParent()->GetOwner() == Owner)))
    {
        return RootComponent->GetAttachParent()->GetOwner()->IsNetRelevantFor(RealViewer, ViewTarget, SrcLocation);
    }
    else if(IsHidden() && (!RootComponent || !RootComponent->IsCollisionEnabled()))
    {
        return false;
    }

    if (!RootComponent)
    {
        UE_LOG(LogNet, Warning, TEXT("Actor %s / %s has no root component in AActor::IsNetRelevantFor. (Make bAlwaysRelevant=true?)"), *GetClass()->GetName(), *GetName());
        return false;
    }

    return !GetDefault<AGameNetworkManager>()->bUseDistanceBasedRelevancy ||
        IsWithinNetRelevancyDistance(SrcLocation);
}

By looking at the code you can tell that setting bAlwaysRelevant to true would be a quite easy fix. In my opinion, this should be the way to go for most actors in small scale maps (maps whom maximum squared distance between two arbitrary actors is less than NetCullDistanceSquared), as actors are generally always relevant to one another, and having them set that way will eliminate the need to check the other conditions including the distance check (this is a server performance killer for higher player counts) at the end, as bAlwaysRelevant is checked first. On the other hand, for large scale maps making every actor bAlwaysRelevant is just rubbing salt in the wound, as no matter how actors are away from each other, they are always relevant, and that’s a huge burden on the server. For that we got the last condition, the net relevancy distance. It’s the same condition that is causing this bug, as the SrcLocation is the viewing location, i.e the location of the SpectatorPawn, which we’ve seen it before as PlayerController.SpawnLocation, and we changed it to be at the player pawn death location. Instead of testing against SrcLocation, we want to test against a location closer to the actor in question (a weapon in our example), and that could be the location of the view target. Therefore, we’ll override the function above and change the last condition to the following:

MyWeaponActor.cpp

bool AMyWeaponActor::IsNetRelevantFor(const AActor* RealViewer, const AActor* ViewTarget, const FVector& SrcLocation) const
{
    ...

    return !GetDefault<AGameNetworkManager>()->bUseDistanceBasedRelevancy ||
        IsWithinNetRelevancyDistance(SrcLocation) || IsWithinNetRelevancyDistance(ViewTarget->GetActorLocation());
}

Note: Pawn and PlayerController classes override AActor::IsNetRelevantFor() and have different conditions for relevancy as a result.

Thus if the simulated proxy in question is a player pawn, you would need to look into APawn::IsNetRelevantFor(), which also has the two conditions we discussed above, so the solution wouldn’t be any different but to override this function this time.

Note: IsNetRelevantFor() is called once (at an interval) on every actor for each connection. Not on every actor for every other actor.

Note: Attaching the SpectatorPawn to the spectated pawn to cure this issue won’t help (unless it’s done to the listen-server player), as the SpectatorPawn exists locally, which is generally client-side. The server should be aware of the changing location of the SpectatorPawn to compute relevancy.

5. Auto Spectate not Working Properly

You may have noticed earlier that dead players default to a static freecam mode, and on a key press they begin spectating alive players. Now what if you wanted that to happen automatically, i.e., camera moves automatically to an alive player without the need to press any key. At which point, you will find yourself running into a brick wall. A failing try could be trying to edit the function that initiates spectating:

MyPlayerController.cpp

void AMyPlayerController::SetPlayerSpectate()
{
    // Only proceed if we're on the server
    if (!HasAuthority())
    {
        return;
    }

    // Update the state on server
    PlayerState->SetIsSpectator(true);
    ChangeState(NAME_Spectating);

    bPlayerIsWaiting = true;

    // Push the state update to the client
    ClientGotoState(NAME_Spectating);

    // View some other alive player
    ViewAPlayer(1);

    // Update the HUD to add the spectator screen
    ClientHUDStateChanged(EHUDState::Spectating);
}

ViewAPlayer(1); was added, and here what this function does:

PlayerController.h

/**
 * View next active player in PlayerArray.
 * @param dir is the direction to go in the array
 */
virtual void ViewAPlayer(int32 dir);

PlayerController.cpp

void APlayerController::ViewAPlayer(int32 dir)
{
    APlayerState* const NextPlayerState = GetNextViewablePlayer(dir);

    if ( NextPlayerState != nullptr )
    {
        SetViewTarget(NextPlayerState);
    }
}

It sets the player’s view target based on the specified direction (+1 is right/next, while -1 is left/previous), exactly like we want. However, you’ll realize that function call doesn’t yield any real result, meaning the player is still stuck in a static freecam mode. Other better, but still unreliable try is to override APlayerController::BeginSpectatingState() to have it call APlayerController::ServerViewNextPlayer().

The reason for such behavior is that we have two roughly parallel execution paths. When a player dies, server will call AController::UnPosses(), either manually or indirectly by calling AMyPlayerController::SetPlayerSpectate():

/// Call stack ///

APlayerController::SetPawn(NULL)
AController::OnUnPossess()
AController::UnPosses()
APlayerController::BeginSpectatingState()
AMyPlayerController::BeginSpectatingState()
APlayerController::ChangeState()
AMyPlayerController::SetPlayerSpectate() // first call

By looking into AController::SetPawn(NULL) you will quickly figure out that it keeps the camera at death location. But that shouldn’t be an issue, considering ViewAPlayer(1) runs at a later point in time, right? Clearly not.

The other parallel (usually slower) execution path is being run by the client, after AController.Pawn is repped down from the server (set inside AController::SetPawn()), triggering AController::OnRep_Pawn() which calls APlayerController::SetPawn(NULL), which again keeps the camera at death location.

The proper easy fix is to override APlayerController::OnRep_Pawn() and trigger spectating after the fact:

MyPlayerController.h

/** Replication Notification Callback */
virtual void OnRep_Pawn() override;

MyPlayerController.cpp

void AMyPlayerController::OnRep_Pawn()
{
    Super::OnRep_Pawn();

    if(IsInState(NAME_Spectating))
    {
        ServerViewNextPlayer();
    }
}

Note that we make sure we are in a spectating state, as that function can run in other scenarios. Also, we fire a server RPC as OnReps fire on client.

6. ADS not Replicating to Spectator’s ViewTarget

Coming soon

System Seems Basic?

While the system is working fine, it may feel for some of you pretty basic. Making it more sophisticated is a matter of choice, and I will leave it for you to do so. Though, I would want to point you to some useful classes/methods that might interest you down the line:

• While we used some quite useful game framework classes, we still haven’t looked into a not less useful one. Considering that this system is heavily camera-based, we have to look into APlayerCameraManager class. You might find there some useful functions you want to use/override, like: AssignViewTarget() and CheckViewTarget().
• APlayerController::GetActorEyesViewPoint(FVector& Location, FRotator& Rotation).
• APlayerController::ServerViewSelf().
• APlayerController::ViewAPlayer(int32 dir).
• APlayerController::GetNextViewablePlayer(int32 dir): The way it works, it lets you view alive players only. Override it if you want to be able to spectate dead players corpses for example.
• AGameModeBase::CanSpectate_Implementation(APlayerController* Viewer, APlayerState* ViewTarget): Controls if player Viewer can spectate player ViewTarget. This can be so useful if you want to dictate a different spectating functionality for different GameModes.