2D Graphics

Most games employ some kind of 2D graphics overlaid on the 3D scene for various purposes which includes heads-up displays, in-game graphical user interfaces (GUI) and menu systems. These overlays are typically comprised of two- and three-dimensional graphics rendered directly in view space or screen space. Overlays are generally rendered after the primary scene, with z-testing disabled to ensure that they appear on top of the three-dimensional scene. Two dimensional overlays are typically implemented by rendering quads (triangle pairs) in screen space using an orthographic projection. Three-dimensional overlays may be rendered using an orthographic projection or via the regular perspective projection with the geometry positioned in view space so that it follows the camera around. The graphical user interface design is a simple and extensible drop-in GUI system that exhibit the following attributes:

• Modular and extensible. The development must encourage the use of design pattern for breaking down a design into modules, which leads to a system to grow cost-efficiently. The code provided should give the user a solid base in which further and more advanced controls may be produced with the least amount of effort.
• Lightweight and flexible. It should be adaptable to as many platforms as possible and the code should be kept as minimal as possible. This guarantees the highest level of portability.
• Object oriented. Object-oriented programming tends to lend itself particularly well to GUI objects with inheritable behaviors and data sets.

The engine defines a GUI system in the BaseUI class (see Graphic/UI/UserInterface.h) that provides an interface for the human view screen elements, and manages all the user controls. Any class that derives from the BaseUI implements an interface to create a particular GUI screen such as, menus or head-up displays (HUD).

class BaseUI : public BaseScreenElement
{
protected:
   eastl::shared_ptr<BaseUIElement> mRoot;
   bool mVisible;

private:
   eastl::map<eastl::wstring, eastl::shared_ptr<BaseUISpriteBank>> mBanks;
   eastl::map<eastl::wstring, eastl::shared_ptr<BaseUIFont>> mFonts;
   eastl::shared_ptr<BaseUIElement> mHovered;
   eastl::shared_ptr<BaseUIElement> mHoveredNoSubelement; 
	
   eastl::shared_ptr<BaseUIElement> mFocus;
   Vector2<int> mLastHoveredMousePos;
   eastl::shared_ptr<BaseUISkin> mCurrentSkin;
};

Any GUI system provides properties such as hot keys, tooltips, context-sensitive help, draggability that can be attached to controls, mostly to give the user a more flexible and informative interface. Currently, the engine has only implemented the tooltip in the GUI system which make it to be aware of moving tooltips around the UI area. It controls that the tooltip is rendered within the allowed area and on top of other controls. The main control of the GUI system is the root which is a special control used to store and group other controls together in hierarchy. It is a container that doesn’t implement any behavior or drawing function, but it provides the entry point towards updating, handling input and drawing all the user controls within. The GUI system provides a factory to create directly GUI-relevant instances like fonts, buttons, or other types and attach them to any parent we pass as parameter. This way we can create straightforward the hierarchy of controls with the root on top of all of them.

The GUI system OnRender function recalculates the root control rectangle each time that the screen size changes, thus it delegates to the root control the process of drawing all their attached user controls. Each control implements the Draw virtual function and some of them may use it to perform calculations each frame. Almost every control draw function makes use of a skin class that helps to modify the look of the UI element.

class UISkin : public BaseUISkin
{
private:
	BaseUI* mUI;

	eastl::array<float, 4> mColors[DC_COUNT];
	eastl::shared_ptr<BaseUIFont> mFonts[DF_COUNT];
	eastl::shared_ptr<BaseUISpriteBank> mSpriteBank;
	eastl::wstring mTexts[DT_COUNT];
	eastl::wstring mIcons[DI_COUNT];
	int mSizes[DS_COUNT];
	bool mUseGradient;

	UISkinThemeType mType;
};

The UISkin class contains all the default control colors, sizes, texts, sprites and icons which are identified by an enumeration index. This way we can access and modify any of the mentioned properties through public functions. Besides, the class implements several drawing functions for displaying rectangles, icons, textures, sprites, background and others variation that are repeatedly being called throughout all the user controllers.

We refer to sprite as the computer graphics term for a two-dimensional bitmap that is integrated into a larger scene. It can be either a static image or an animated graphic to simulate some form of motion or to achieve certain special effects. In the engine we use sprites to draw static images for different button states, such as (un)pressed, (un)focused, mouse over/off, and to draw icons for checkboxes whenever they indicate an "on" or "off" state via a check mark/tick ☑ or a cross ☒. Sprites are mechanism to animate and draw entities in order to make user interface elements dynamic and visually appealing. The engine uses a simple keyframe system of textures that has the advantage of simplifying the UI control in comparison with more advanced system of animation which uses timelines, key frames, transformations and interpolation curves to create complex animations. The disadvantages of this system are that it may lack in detailed animation frames causing artists to work harder for creating extensive or highly detailed user interface, and that the overall video memory cost is higher.

In the system, we have implemented a bank of sprites which keeps an instance of every sprite that is drawn on screen in order to avoid duplication of assets in memory. As the following code shows, the AddTextureAsSprite function in the UISpriteBank class adds the texture and use it for a single non animated sprite. We identify the registered sprite with an index value assigned increasingly every time we call this function. This allows us to create an enumeration for default icons (UIDefaultIcon in the UISkin class), that has the most common UI icons we can find in any application.

   eastl::shared_ptr<ResHandle>& resHandle = ResCache::Get()->GetHandle(
   &BaseResource(mCurrentSkin->GetIcon(DI_WINDOW_MAXIMIZE)));
   if (resHandle)
   {
      const eastl::shared_ptr<ImageResourceExtraData>& extra =
      eastl::static_pointer_cast<ImageResourceExtraData>(resHandle->GetExtra)
      extra->GetImage()->AutogenerateMipmaps();
      extra->GetImage()->SetName(mCurrentSkin->GetIcon(DI_WINDOW_MAXIMIZE));
      mCurrentSkin->GetSpriteBank()->AddTextureAsSprite(extra->GetImage());
   }

Whenever we want to draw a specific icon, we call Draw2DSprite function in the UISpriteBank class and pass the index of the icon to be drawn along with the other parameters. The sprite bank class holds the information required to perform a sprite’s animation defined in the UISprite struct. This UISprite struct contains the frames that compose the entire animation and the time that dictates the frame changing rate. Both information are used for keeping the animation state of a sprite, as well as for advancing the animation track at the specified frame rate.

Another important property of the skin class are fonts. A computer font is implemented as a digital data containing a set of graphically related glyphs, characters, or symbols. It is generally used to refer to a set of digital shapes in a single style, scalable to different sizes. A font family or typeface refers to the collection of related fonts across styles and sizes. There are three basic kinds of computer font file data formats:

• Bitmap fonts consist of a matrix of dots or pixels representing the image of each glyph in each face and size.
• Vector fonts (including, and sometimes standing as a synonym for outline fonts) use Bézier curves, drawing instructions and mathematical formula to describe each glyph, which make the character outlines scalable to any size.
• Stroke fonts use a series of specified lines and additional information to define the profile, or size and shape of the line in a specific face, which together describe the appearance of the glyph.

The engine uses bitmap fonts which are faster and easier to use in computer code, but they are non-scalable that requires a separation font for each size. Outline and stroke fonts can be resized using a single font and substituting different measurements for components of each glyph, but are somewhat more complicated to render on screen than bitmap fonts. They require of additional computer code for rendering the outline to a bitmap and display it on screen. Although all types are still in use, most fonts seen and used on computers are outline fonts. The UIFont is a very useful class that wraps the necessary functionality to load font, renders text using different fonts, and calculates the extents of the text using the function GetDimension. The GUI system is responsible of handling all the available fonts and load them on request, that is, if it doesn’t exist.

    eastl::shared_ptr<BaseUIFont> font(GetFont(L"DefaultFont"));
    if (font) skin->SetFont(font);

    BaseUIFontBitmap* bitfont = 0;
    if (font && font->GetType() == FT_BITMAP)
	bitfont = (BaseUIFontBitmap*)font.get();

    skin->SetFont(font);

    eastl::shared_ptr<BaseUISpriteBank> bank = 0;
    if (bitfont)
	bank = bitfont->GetSpriteBank();
    skin->SetSpriteBank(bank);

There are basically two ways to get text displayed on the screen. The first option is to add a static text through the GUI system factory which uses the built-in font and can only be changed through the overriden font function. The second option is to create a font object using the GUI system factory and specifying a font that we want to be loaded into our application. In both cases we can draw a text on the screen using the Draw() method of the UIFont object.

GUI systems are required to perform more than animating sprites and displaying text. All controls may potentially handle some user input, which the GUI system will forward after it has processed the message in OnMsgProc function. The OnMsgProc implementation will detect if controls should come into focus, or if the mouse may be hovering over the control, or is being selected when is clicked onto the element. When any of these actions occur, it will send out the event to the element we want to apply the action by calling the control virtual function OnEvent, so it can update itself and propagate the event message through its parents. If the event was handled, it returns true, otherwise it returns false. This allows us to chain input handling calls and stop when any has successfully handled the input.

Each user control implemented in the engine (buttons, labels…) needs to have a way to pass new events to the game user interfaces (menus, HUDs) which may occur while processing the input. Modern engines implement event handlers which are mechanisms to create reusable, general purpose callbacks to respond the events generated by the user control while still maintaining the system decoupled. However, we have created a simple and centralized mechanism which has GUI dependencies in order to establish communication between the engine user controls and game user interfaces. The control generates the new event and propagates it up to their parents until it reaches the root control which is the responsible for forwarding UI events to the game user interfaces. In response to the event, any game user interface can implement its own behavior in the centralized OnEvent function from the BaseUI class.

    Event newEvent;
    newEvent.mEventType = ET_UI_EVENT;
    newEvent.mUIEvent.mCaller = this;
    newEvent.mUIEvent.mElement = 0;
    newEvent.mUIEvent.mEventType = UIEVT_BUTTON_CLICKED;
    mParent->OnEvent(newEvent);

When processing input messages, there are two distinct cases we are testing for, the first one is if the mouse is within the control or not. For this purpose, we use the hit testing algorithm which checks the control’s and all of its descendant’s rectangle boundary (see function GetElementFromPoint). The second case is when the control comes into focus, this is done in the user interface SetFocus function by some interaction from the user. If the user presses within the control’s rectangle, the control will come into focus and we will invoke the OnEvent control function to inform the element that it has been focused, otherwise if the button press happened outside the control (or on another control) we will invoke OnEvent control function to inform the element that it has lost the focus. The user interface OnPostRender function will take care of updating a visible tooltip from any hovered control and also delegates the post-rendering to the root control.

In the engine, the graphical user controls that we have explained are also known as interface elements. They are software components that a user interacts with through direct manipulation to read or edit information about an application. There are many well-known user interface elements that we are familiar with and use frequently to perform very precise functions. The goal is to develop a working library of controls that are general enough to cover many situations, and are scalable to easily add more game-specific controls.

class BaseUIElement : public eastl::enable_shared_from_this<BaseUIElement>
{
public:
	virtual void OnInit();
	virtual bool OnEvent(const Event& event);
	virtual void Draw();
        virtual void OnPostRender(unsigned int timeMs);

        eastl::shared_ptr<BaseUIElement> GetElementFromPoint(const Vector2<int>& point);

protected:
	eastl::list<eastl::shared_ptr<BaseUIElement>> mChildren;
	eastl::shared_ptr<BaseUIElement> mParent;
	RectangleShape<2, int> mRelativeRect, mAbsoluteRect, mDesiredRect;
	RectangleShape<2, float> mScaleRect;
	Vector2<unsigned int> mMaxSize, mMinSize;
	bool mVisible, mEnabled;
	eastl::string mName;
	int mID;
	UIAlignment mAlignLeft, mAlignRight, mAlignTop, mAlignBottom;
	UIElementType mType;
}

The BaseUIElement is the base control which provides the interface that any derived controls may implement to offer their own behavior. They implement the following abstract functions; OnInit() initialize the control, OnEvent() receive and process any system event and update the control in response, and the draw functions for rendering the control and optionally do some post rendering operations. Some amount of functionality is also shared among the majority of controls such as hit testing implemented at the base class level in the GetElementFromPoint function.

Most controls will share the same core set of features regardless of what the control’s intended purpose is such as, identification, placement, and other states which we will explain next. The base control contains common variables that determine whether the object is considered active and whether it is visible. The distinction here is that an object could be visible on screen, but not active in that it does not respond to input or update itself. Likewise, an object could be actively being processed, but not visible on-screen. Every control needs an identification to distinguish itself uniquely from the other controls on the screen and a human-readable name stored as string. The base control also contains the parent/children relationship of the user interface hierarchy. A control can have any number of child controls but only one parent. Controls on the same level are considered siblings, a control placed a level above relative to the current control is called parent, and a control placed a level beneath to the current control is called a child.

All controls will use a rectangle to define their bounds on screen, these bounds are useful for calculating placement and performing hit testing to determine if the control needs to perform some action due to input. The bounds define the position and size in pixels of the control which are recalculated depending on the alignment parameters. It is employed two types of rectangle; we will define one to be the desired rectangle or a template rectangle that the control would prefer to be without size constraints. The others are floating rectangles calculated from the desired rectangle after applying size constraint in their own coordinate space, which are frequently accessed for positioning, in relative and absolute coordinates.

When working with user interfaces, one thing that comes up often is the need to align elements to anothers, referring to the top, bottom, right and left space position relative to its parent control. We apply the alignment to the rectangles in the RecalculateAbsolutePosition function which is driven by a set of straightforward equations that modify the position and size of one element respect to another. The engine has enumeration flags that will allow us to combine different alignments when needed, we would call SetAlignment(left, right, top, bottom) function that defines how the borders of the control will be positioned when the parent element is resized. For example, in order to get the control aligned to the center right from its parent, we would combine the flags SetAlignment( UIA_LOWERRIGHT, UIA_LOWERRIGHT, UIA_UPPERLEFT, UIA_LOWERRIGHT ). The left and right border of the control are placed to the lower right position from its parent, and the top and bottom border of the control are placed in the corners from its parent in order to center it. The goal is to provide an easy way to apply the alignment action that we may want to do often, and we must provide it in an intuitive, unobtrusive way. It is also important to provide the means to refresh a control since it is not necessary for controls to calculate its placement coordinates every frame. Whenever is applied any significant operation onto the element such us, change the rectangle area, the position or the size constraints, it will be recalculated recursively the control bound and their children using UpdateAbsolutePosition().

//! Updates the absolute position.
virtual void UpdateAbsolutePosition()
{
   RecalculateAbsolutePosition(false);

   // update all children
   eastl::list<eastl::shared_ptr<BaseUIElement>>::iteratorit=mChildren.begin
   for (; it != mChildren.end(); ++it)
   {
      (*it)->UpdateAbsolutePosition();
   }
}

Each control which has been implemented in the engine facilitates a specific user interaction. We will define every single one of them, describe their main features, and point out in which context are they used for.

• Button. There are many different types of buttons, but the majority share a common set of features. A button is a user interface element that has different states, its default state is when the button is not pressed and ready for input. It can be active, or selected, which could mean the presence of some input device is over it, like a mouse cursor. When input is received, the button is pressed and at some point, it will be released. Some buttons may also have some additional properties, such as being disabled to prevent any input from the player. There are some variations of the button which can be implemented:
  • Radio button. Control which can be clicked upon to select one option from a selection of options. Radio buttons always appear in pairs or larger groups, and only one option in the group can be selected at a time; selecting a new item from the group's buttons also de-selects the previously selected button.
  • Check box. Control which can be clicked upon to enable or disable an option. Also called a tick box. The box indicates an "on" or "off" state via a check mark/tick ☑ or a cross ☒. Can be shown in an intermediate state (shaded) to indicate that various objects in a multiple selection have different values for the property represented by the check box. Multiple check boxes in a group may be selected, in contrast with radio buttons.
  • Split button. Control combining a button (typically invoking some default action) and a drop-down list with related, secondary actions.
• Slider. A slider is a useful numeric input control that allows players to select between a fixed range of values. The most common place where we see slider controls is in sound configurations to control the volume of sound effects and music, but they are also used to control gamma values to calibrate displays. It can be moved up and down (vertical slider) or right and left (horizontal slider) on a bar to select a value (or a range if two handles are present). The bar allows users to make adjustments to a value or process throughout a range of allowed values. A slider’s look can vary depending on the game style, but it usually will consist of a background, a cursor, a label to represent the minimum value, a label for the maximum value and a label for the current value of the slider. The position of the labels is purely cosmetic and entirely dependent on the look and feel of the game, the behavior of a slider is very similar to the scroll bar.
• Scrollbar. A graphical control element by which continuous text, pictures, or any other content can be scrolled in a predetermined direction (up, down, left, or right). A scroll bar is a control that is useful in conjunction with other controls, namely with controls that have so many options that cannot be visible within the designated space. The scroll bar controls the visible elements or area of some other control. The behavior of a scroll bar can be abstracted to work as a standalone entity that other controls use. The scroll bar will be constructed with some knowledge of the control it is linked to, its size, how many options and the size of each option. The scroll bar can be manipulated by users, scrolling between a minimum point and a maximum point. To the control that owns the scroll bar, it is the ratio, the value that represents the relative position between the minimum and maximum points that will be useful.
• Tree view. A graphical control element that presents a hierarchical view of information. Each item can have a number of subitems, often visualized by indentation in a list. An item can be expanded to reveal subitems, if any exist, and collapsed to hide subitems.
• List box. A graphical control element that allows the user to select one or more items from a list contained within a static, multiple line text box. The simplest case to illustrate a list box’s functionality is using text strings as the items in the list. However, to give list boxes more flexibility, we have implemented it so that each item in the listbox is a user interface control, by doing this we can create list boxes of more complex items, such as game elements represented by an image and text, even include the item’s description directly in the list box.
• Dropdown Box. It is a compact container that may hold multiple options yet only one of them may be selected at a time. The selected option is visible while the remaining options are hidden and only visible when the user activates the dropdown menu.
• Combo box. A graphical control element combining a drop-down list or list box and a single-line editable textbox, allowing the user to either type a value directly into the control or choose from the list of existing options.
• Text Box. A graphical control element intended to enable the user to input text. Text boxes are the way to allow data entry through a keyboard, this means that when anyone type on a physical keyboard or a virtual one on a portable device, the tapped key must immediately display the corresponding character within it. Next, typing must be instantaneous, some people can type very fast, any artificial delay in the text boxes input is going to severely hinder the user’s experience and will give the impression that the software is running slowly; it must not, however, be so fast that holding a key for a fraction of a second would display the character multiple times. This means we need to detect a long key press and, in this case, provide a small delay and then accelerate the rate of character entry, this is especially true for backspace, delete and the space bar. Another thing that is expected during text entry is a caret, typically the caret is a thin pipe that blinks to represent the location at which the next character will be displayed, in some cases it may be a small square or some other shape that may depend on the art direction. There is a reset parameter used when the auto complete text may have changed enough that the prefix is no longer valid, this can happen when the user presses backspace and erases a part of the prefix.
• Label. A label is used as a way to display text, it cannot be interacted with and has no actual functionality except to convey information. Labels provide us with a text field on which to display information to users, the information can be static, meaning the label does not change; for example, when displaying titles or captions. Or, it can be dynamic, the content of the label may be a game variable, scores, player names or an active game state, a dynamic label’s value will be provided by some external source and may change every frame.
• Window. A graphical control element consisting of a visual area containing some of the graphical user interface elements of the program it belongs to. Windows are used for important parts of the game such as menus or dialog boxes.
  • A menu is a user control with multiple actions which can be clicked upon to choose a selection to activate. As a user interface programmer, it is important to identify the parts that all menus have in common and create a base class that handles the default behavior of any menu and also provide the means to create new menus and change their behavior based on the requirements of the game’s direction. Most game menus require the use of a variety of controls to change game’s settings such as sound and music volumes, gamma correction, brightness controls, character customization options, etc. They must respond to different types of input, mouse input or touch input as well as keyboard or gamepad navigation.
  • A dialog box is a small window that communicates information to the user and prompts for a response. It’s useful to have a way to communicate with the user, whether it’s the player or the developer that something has happened. The message box design is usually modal, meaning that the user is required to interact with it before the game can return to its previous state.

As example, we will describe the steps to implement a functional and working button. The UIButton class (see Graphic/UI/Element/UIButton.h) inherit from the BaseUIButton class which is an abstract class that derives from the BaseUIElement class. Besides of the common BaseUIElement functions for initializing, handling events and drawing, the button will inherit from the BaseUIButton functionality in terms of determining what kind of button it is, in which state it is, focused, hovered over, or pressed, and other functions to support its drawing.

class UIButton : public BaseUIButton
{
	struct ButtonSprite
	{
		int Index;
		eastl::array<float, 4> Color;
		bool Loop;
	};

	ButtonSprite mButtonSprites[BS_COUNT];

	eastl::shared_ptr<BaseUISpriteBank> mSpriteBank;
	eastl::shared_ptr<BaseUIFont> mOverrideFont;

	eastl::shared_ptr<Texture2> mImage, mPressedImage;
	RectangleShape<2, int> mImageRect, mPressedImageRect;

	unsigned int mClickTime, mHoverTime, mFocusTime;
	bool mPushButton, mPressed;
};

A button has its own input handling to do, which are press and release. Typically, a button will perform its intended function upon its release, but also knowing that a button has been pressed may be useful to provide feedback to the player. As we have explained previously how GUI System communicate events, when the users release a button it will send a new event to its parent in order to let the game user interface implement its own behaviors as a result. The GUI system already gives us information when the mouse cursor is over its bounds, and we can take advantage of this information to avoid duplicating the boundary checking and determine if a button has been pressed. We are only interested in that state in order to avoid notifying the button press while the button is held down. To determine if the button has been released, we first check if the button is in the pressed state, if so, then we can revert the button’s state back to not pressed and send an event to the GUI system to report about this change of state. There are cases we may want to know if the mouse is held down, if so, we set the focus on the control after checking that it doesn’t have the focus on it and that the mouse is over the button boundaries. To draw our button, we will simply call the texture draw function of the skin class, to which we provide the texture visual effect and the dimension required to draw the button. We can make buttons more visually interesting by using sprites to render different animations depending on the state of a button. The last part is to draw any text associated with the button for what the UIFont class is responsible. We need to pass to the draw function the text’s rectangle in a way that it will fit within the button. By default, the text is aligned to the center of the button and the color is gray out if the control is disabled, otherwise it is specified in the UIDefaultColor enumerator for button texts.