Storyboard Architecture
There are three fundamental Storyboard concepts that you must understand when building user interface applications with Storyboard: The application model hierarchy, the event based interaction, and using variables for manipulating dynamic content. Having a solid grasp of these fundamentals will guide you from creative concept to implementation.
Application Model Hierarchy
An application is organized as a hierarchy of screens, layers, groups, and controls that are collectively called model elements.
Screens are used to represent contextual visual states of your application. When working within Designer your primary viewport into how the application is organized will be through the screens of the application that are defined. Structurally a screen is an ordered list of layers that are composited together to form the final screen display. In a Storyboard application only one screen can be active at a time.
Layers are used to organize related visual elements together in a reusable manner. In this case, reuse means reusable in different screen contexts so that the same layer can be associated with several different screens. When layers are shared among multiple screens, for example, as a common menu or background, then the content of the layer is exactly the same in those different screen contexts. If content changes on the layer in one screen context, then the content will change across all screens that use that layer. While the content for layers may be shared, there are several properties that are tied specifically to how the layer is used within a screen. These properties are called layer instance properties since they relate directly to a particular instance of a layer and include the x/y position of the layer, the visibility/hidden state, and the transparency or alpha blending value of the layer.
Since the layers have a configured size, content can be positioned within the layer and will clip to the bounds of the layer and not draw outside of it. By adjusting the position of a layer, you can move a significant amount of visual content with minimal effort. Layers however, are still containers for other structural objects, groups, and controls.
Controls are the primary user interaction points in a Storyboard application. Each control defines a clipping region and contains an ordered list of render extensions, which are the base rendering and drawing primitives. These include images, text, fills, arcs, and polygons (and more!). All rendering performed by a control's extensions occurs within the bounds of the control.
Controls can also contain other controls, allowing for the creation of complex, reusable UI components. This nested structure supports encapsulation of layout, behavior, and visual presentation.
Each control has properties that define its position, size, and layout behavior. These properties include:
-
grd_x,grd_y- fixed offsets from the parent’s origin. -
grd_sx,grd_sy- scaled offsets as a fraction of the parent’s width or height. -
grd_w,grd_h- fixed width and height. -
grd_sw,grd_sh- scaled width and height relative to the parent. -
grd_ax,grd_ay- anchor points that determine how the control’s position is interpreted (LEFT/CENTER/RIGHT or TOP/CENTER/BOTTOM). -
grd_pl,grd_pt,grd_pr,grd_pb- padding values that define the client area for child controls.
The resolved position of a control is calculated by combining its fixed and scaled offsets:
Resolved_X = Fixed X + Parent_Width * Scaled X Resolved_Y = Fixed Y + Parent_Height * Scaled Y Resolved_Width = Fixed Width + Parent_Width * Scaled Width Resolved_Height = Fixed Height + Parent_Height * Scaled Height
Anchors define how the resolved position is interpreted relative to the control’s bounds.
Padding defines the internal margins of a control and determines the client area available for its children. This allows for precise layout control and consistent spacing within nested structures.
Controls can be used individually or in combination to achieve a desired visual effect. A single control may use multiple render extensions stacked within its bounds, or multiple controls may be composed together to form a unified visual component. Grouping controls can help organize related elements, with nested controls, layout cohesion can also be achieved directly within the control hierarchy.
Groups are a collection of controls. Unlike a layer however, a group does not clip its content and the size of the group is determined by the bounds of the controls it contains. A group does have a position and all of the controls within the group will have a position that is relative to the group's position. In this way, groups are similar to layers in that changing the position of a group will translate all of the controls at once or changing the visible/hidden state will cause all of the controls contained to either be shown or hidden.
Groups and controls are both child objects of a layer, but groups can only contain controls. There is currently no grouping of groups, though that may be a future consideration.
Tables are a special form of templated control that is repeated over and over in a grid or list like pattern. Tables and their table cell control templates are treated the same as controls, but use a different representation for dynamic data. This topic is covered in more detail in the chapter Creating Lists and Tables.
The application is the root of this hierarchy. The application maintains the list of all screens, layers, groups, and controls and determines at the start of execution which screen is the first displayed. The start screen setting is a property of the screen model object and can be set within Designer. Every application must have at least one screen and every screen must have at least one layer to be valid. Designer will generally raise an error on export if these conditions aren't met within the application design.
Event Driven Interaction Model
Storyboard is an event-driven user interface state machine. Events are the basic communication mechanism for triggering activity and passing data in a Storyboard application.
Events are named entities and optionally carry an extra binary data payload. Events come in three main flavors:
-
Traditional events such as press, release, motion, and keydown are generated by standard input sources. Storyboard provides a public definition for these common events so that they can be externally generated in a consistent manner.
-
Application events are specifically related to the user interface and trigger activity from within it. Events used this way form an internal API for the application and are generally private as they may be used solely for triggering a common UI experience such as
start_animationorgoto_screen. -
External events typically make up the data communication API with a backend service that contains the core data model of the product. These events frequently contain a domain-specific data payload related to the application. External events represent the API between the front-end user interface and backend service and are specific to a product and should be well-defined to support parallel development between front-end and backend developers.
Events may be received from multiple different input sources, but are always processed serially by the Storyboard Engine.
Events trigger actions. Actions perform tasks, such as manipulate data, interact with the system, log messages, and generate more events. There are many different actions that perform different things. Screen transition actions perform visual screen transitions. Data change and animation actions modify data within the UI that can cause a visual change. Timer actions trigger new events in either a repetitive or one-shot manner. Lua script actions invoke a callback function scripted in Lua (Lua is a script programming language used for glue logic in Storyboard). In addition to all of the fixed actions, the Storyboard SDK provides users with the ability to write and contribute their own actions to the engine.
The event/action association can be made on any of the model elements. For example, a control with an image render extension that is acting like a push button might have an event/action association that would match a press event and cause a data change action that hides or shows a control or changes the value of a variable.
Where the event/action association is made is important. Associations that are made on controls must be triggered by position-related events, such as press and release, otherwise the control must use focus settings to ensure that non-position based events (such as external events) are delivered to their context. Associations that are made on layer or group model objects are more generic, but in order to be triggered the event must first be potentially triggered against a control in that context. Screen actions will always be matched as long as the screen is the current visible screen. Application event/action bindings are the most general and will match in all contexts and are generally the most suitable when matching external events and processing model data.
The event-driven model that Storyboard uses for user interaction has significant benefits that may not be immediately apparent. The first is that the events form a discrete and defined interface point between the UI and the rest of the product. By isolating the external events:
-
Graphic designers have a clear expectation regarding the data they have available to interpret.
-
Test developers can exercise both the front end and backend by generating simulated events with artificial data.
-
Backend software developers have a clear direction on what data they are making available and do not have to overexpose or overdevelop their interfaces for functionality that will never be used.
The event model also ensures a complete decoupling of the UI from the implementation and is what makes desktop simulation a practical reality. Simulated data can be easily injected in environments where the real target hardware may not be ready, prohibitively expensive, or just plain awkward to work with. Building the UI in a comfortable environment and knowing that it will move readily to the embedded target at the time of deployment with small adjustments is an invaluable boost to overall productivity.
Events to external applications
Communication with external processes in the embedded system can be accomplished in several ways. One approach that provides a strong API while maintaining a loose coupling for the implementation is to use Storyboard IO.
Storyboard IO is provided as a plugin for the Storyboard Engine and includes a C API and library for external applications to link against.
When the Storyboard IO plug-in is loaded a channel is created in order for processes to inject events into the system. A single event queue is used to serialize the events and therefore any events sent via Storyboard IO will be placed in the queue with standard Storyboard system events. If the external application wishes to receive events, it can create its own Storyboard IO channel which can have events sent through. Applications can have multiple receive channels and the Engine has a single input channel. The following diagram illustrates an application which can send events to the Engine and review events on a named channel.
Sending and Receiving Data with Storyboard IO discusses how to work with Storyboard IO to send and receive events.
Data Binding for Dynamic Behavior
Render extensions associated with controls render the content, from text and images to arcs and polygons. Each render extension's behavior is controlled through a set of properties that the user can configure, such as fill color, text size and font, and image filename. By default, the settings for these properties are fixed at design time and can't be changed dynamically at runtime. However, these settings can be dynamically adjusted at runtime by using Storyboard variables.
Attributes, for both render extensions and actions, that may require dynamic adjustment at runtime can be associated to named Storyboard variables.
Similar to the event/action association, variables can be bound to any of the model elements in the application. This ownership dictates how that variable will be referenced when it is associated with a property in a render extension or action. Storyboard variables are managed as loosely typed key/value pairs in the Storyboard Engine's data manager component. The key is the fully qualified name of the variable as it is declared within the model and the value is the value of the variable set initially within Designer and changeable at runtime through various actions, animations, or using the Lua API.
The Storyboard model is hierarchical, so the construction of a fully
qualified model path is a straightforward process of joining model
element name segments with a . (dot) in between them. The following
list demonstrates how the fully qualified model name is formed for a
variable, varname, associated with different contexts in the model.
varname | This identifies a variable, |
screen_name.varname | This identifies a variable, |
layer_name.varname | This identifies a variable, |
screen_name.layer_name.varname | This identifies a variable, |
layer_name.control_name.varname | This identifies a variable, |
layer_name.group_name.control_name.varname | This identifies a variable, |
There is some overlap in the Storyboard namespace that could lead to ambiguous resolution. To maintain a clear name resolution, layers and screens may not have the same names. Within a container, such as a layer or a group, all of the model element names must be unique. Storyboard Designer enforces this restriction.
Which model element owns a variable is usually a good indication of its scope. Variables that are owned by control model elements will typically have an association with render extensions or actions local to that control. A variable with a broader ownership, such as at the screen or application level, will likely have several different associations within the model.
For example, to represent a button with a changing image, one might create a variable on a control that is used by an image render extension on that control. To easily configure the fonts and text sizes used within the application, you might create several variables at the application level and have all of text render extensions refer to those variables.
Variables represent data that is used within the user interface but are not tied to any single context. The same variable may be referenced by different render extension clients or even associated with action properties.
When you want to make a change to the visual presentation of the user interface, you don't perform drawing calls directly, instead you make changes to one of the variables that you have created. The Storyboard Engine monitors the values of all variables. As variable content changes, Storyboard determines if the variable is used by the current on-screen content. If there are changes, the Engine automatically schedules a refresh operation and groups the refresh operations together for efficiency.
To associate a property with a variable or to change the value of a variable, it's important to understand how a variable is addressed within the application. The addressing scheme generally follows the model hierarchy. When a variable is sourced from the application root using this full path, it is called a fully qualified name.
Note that variables associated with layers and variables associated with screens have the same addressing path. This is because a layer and it's content can exist on multiple screens, and removing the screen component allows a consistent address path to groups and controls on that layer.
Context and Variables
It can be cumbersome to always use a variable's fully qualified name. In some cases it is more convenient to address a variable by its context of use. For example, render extensions are always associated with a control and in many instances the properties that they associate to variables will also be local to that control. In this situation it would be more convenient to refer to a variable as being 'local to the control'. Storyboard supports this through the use of a data context shorthand that can replace a variable's fully qualified name.
The data context is the context of execution of a particular activity.
For render extensions, this means the parent control of the render
extension. For actions, this means the model element associated with the
triggering event. The shorthand notation is a string formatted as
${@@CONTEXT:NAME} where CONTEXT is going to be one of app, screen,
group, layer, or control indicating the model element in context to
resolve, while the NAME is the name of the variable being addressed.
To reduce complexity of using the fully qualified model paths and to minimize the maintenance effort as a project evolves or changes, Storyboard defines several variable shortcuts that will expand their value based on the current model context in which they are being resolved.
| Refers to the application variable |
| Refers to the screen variable |
| Refers to the layer variable |
| Refers to the group variable |
| Refers to the control variable |
| Refers to the event variable |
For example, in the following Storyboard model:
Application
- MainScreen
- ALayer
- AGroup
- AControl
- AGroup
- ALayer
where the current focus is associated with the control AControl,
reference to a variable varname would resolve to a fully qualified
path as follows:
|
|
|
|
|
|
|
|
|
|
For tables, where the context also includes the row and column index of
the table cell being adjusted, a special ${cell:CONTEXT:NAME} notation
can be used that converts the variable shorthand into a variable that
contains the row and column as a postfix to the standard variable name.
For example, if the current focus or selection was directed a table
control named ATable on the cell that was occupying row five and
column one then the following would be variable expansions:
|
|
|
|
|
|
Storyboard Model Internal Variables
Not all variables have to be defined by the user and associated with
render extension and action properties. Storyboard defines a set of
variables for the model element properties such as position, size, and
visibility. Each class of model elements contains a slightly different
set of values, but all of the variables are prefixed with the grd_
(Graphical Runtime Data) variable namespace to avoid any confusion with
user created variables.
These variables are generally accessed using ${model_object:varname},
for example ${control:grd_x} indicates the x position of the current
control.
Layer and Layer Instance Variables
The following values can be queried and changed through normal data management channels. The following values can be queried and changed through normal data management channels. If these variables are being changed directly the fully qualified path, including the screen on which the layer resides (<screen>;<layer>;<variable>;), should be used.
| 2s1 | The layer instance's x position relative to the screen |
| 2s1 | The layer instance's y position relative to the screen |
| 2s1 | The scaled component of a layer instance's x-position, a factor of the application's width inherited by this layer instance. |
| 2s1 | The scaled component of a layer instance's y-position, a factor of the application's height inherited by this layer instance. |
| 1u1 | The layer instance's horizontal anchor. Determines how a layer instance's x-position is interpreted, defining which point of the layer instance is anchored.
|
| 1u1 | The layer instance's vertical anchor. Determines how a layer instance's y-position is interpreted, defining which point of the layer instance is anchored.
|
| 4s1 | The x pixel offset that will be used to determine the origin of the layer instance |
| 4s1 | The y pixel offset that will be used to determine the origin of the layer instance |
| 1u1 | The layer's transparency value. The values range from 255 (opaque) to 0 (transparent) |
| 1u1 | The layer's visibility. A value of 0 states that the layer and all of its controls are visible and a value of 1 hides the layer and all of its controls |
| 1u1 | The layer instance's content scrolling enablement. A value of 0 indicates that the layer can not be scrolled and a value of 1 indicates that the layer can be scrolled. This value can only be modified on layers that have had scrolling enabled in Storyboard Designer. |
Any change to the following values affects all layer instances.
| 2s1 | The layer's width |
| 2s1 | The layer's height |
| 2s1 | The scaled component of a layer's width, a factor of the application's width inherited by this layer. |
| 2s1 | The scaled component of a layer's height, a factor of the application's height inherited by this layer. |
Group variables
The following values can be queried and changed through normal data management channels.
| 4s1 | The group's x position relative to its layer |
| 4s1 | The group's y position relative to its layer |
| 4s1 | The group's z-index position. This sets the stacking order of groups within its layer where 0 is at the back (furthest from the eye). |
| lu1 | The group's visibility. A value of 0 indicates that the control is visible and 1 that it is hidden |
Control variables
The following values can be queried and changed through normal data management channels.
| 2s1 | The control's x position relative to its layer |
| 2s1 | The control's y position relative to its layer |
| 2s1 | The control's width (same as grd_width) |
| 2s1 | The control's height (same as grd_height) |
| 2s1 | The control's width (same as grd_w) |
| 2s1 | The control's height (same as grd_h) |
| 4f1 | The scaled component of a control's x-position, a factor of the parent's width inherited by this control. |
| 4f1 | The scaled component of a control's y-position, a factor of the parent's height inherited by this control. |
| 4f1 | The scaled component of a control's width, a factor of the parent's width inherited by this control. |
| 4f1 | The scaled component of a control's height, a factor of the parent's height inherited by this control. |
| 1u1 | The control's horizontal anchor. Determines how a control's x-position is interpreted, defining which point of the control is anchored to the specified x coordinate.
|
| 1u1 | The control's vertical anchor. Determines how a control's y-position is interpreted, defining which point of the control is anchored to the specified y coordinate.
|
| 2s1 | The control's left padding, used to determine the client area for children of this control. |
| 2s1 | The control's top padding, used to determine the client area for children of this control. |
| 2s1 | The control's rightpadding, used to determine the client area for children of this control. |
| 2s1 | The control's bottom padding, used to determine the client area for children of this control. |
| 4s1 | The control's z-index position. This sets the stacking order of controls within its layer where 0 is at the back (furthest from the eye). |
| 1u1 | The control's visibility. A value of 0 indicates that the control is visible and 1 that it is hidden |
| 1u1 | A value of 1 states that the control is active (can receive and react to events) and 0 for an inactive control (cannot receive or react to events) |
| 1u1 | Indicates if the control is opaque to events. If opaque (1), the control will block events from being handled by other controls. If the value is 0, the events flow through the control to ones behind it. |
| 4s1 | The control's focus index. This sets the focus on a control in a navigation sequence, where 1 sets the focus on the first control, 2 sets the second, etc. A value of 0 indicates that the control is not focusable. In order for a control's focus index to be changed dynamically at runtime, the focus value must be initially set to a non-zero value in Storyboard Designer. |
| 1u1 | If a mask render extension is present, this value indicates that a mask rendering behavior should be applied to the control. |
| 1u1 | Indicates if a control's childrenc can draw outside the bounds of this control. The clipping specified by the this control's parents is still respected. This does not apply to render extensions that are part of this control. |
Table variables
A table contains all of the control variables and also a set of table
specific variables. These table specific variables can be queried but
not dynamically changed. In order to change these values in a table,
actions are provided: gra.table.resize, gra.table.scroll. The
variables are as follows.
| 4s1 | The number of rows in the table |
| 4s1 | The number of columns in the table |
| 4s1 | The number of visible rows in the table |
| 4s1 | The number of visible columns in the table |
| 4s1 | The row index of the currently active cell |
| 4s1 | The column index of the currently active cell |
| 4s1 | The table’s current top left row |
| 4s1 | The table’s current top left column |
| 4s1 | The x pixel offset that will be used to determine the origin of the 1,1 table cell |
| 4s1 | The y pixel offset that will be used to determine the origin of the 1,1 table cell |
| 1u1 | The table's content scrolling enablement. A value of 0 indicates that the table can not be scrolled and a value of 1 indicates that the table can be scrolled. This value can only be modified on tables that have had scrolling enabled in Storyboard Designer. |
Animation Definitions
Animations can have an enormous impact on the application's user experience and are an important part of any modern user interface. Used well, they can make a relatively dull user interface appear modern and intriguing. Used poorly, they can waste a user's time and ruin what might have been a highly useful application.
An animation in Storyboard manipulates variables exposed by the application model. These variables may be the user-defined variables or they may be the internal variables associated with model element properties. When the variables are numeric there are a variety of easing functions that can be applied to transition from one value to another. Many variable changes can be orchestrated to occur at points relative to one another and edited on a timeline within the design environment and ultimately saved into a named animation block that accompanies the exported model.
A specific animation can be created that relies on the fully qualified paths to specific variables or a generic animation can be created that relies on variables that use the context relative naming convention. In both situations, the definition of an animation is independent from its use.
In order to use or trigger an animation, the definition has to be referenced by an action, which in turn is associated with an event. The Animation Action is specifically intended for the purpose of launching user-defined animations designed in Storyboard Designer.
When the visual content of an application is dynamically created, it may not be practical to create animations within Designer in advance. In these situations it is possible to create rich animations programmatically using the Lua Script Action.