Wallframe Infrastructure

WallFrame consists of several component processes that work in tandem to provide a framework for application development. The over-arching idea is that for compelling applications on large scale displays, developers need to bring together many different functions and capabilities. Our goal is to compartmentalize these different functions into discrete processes, and then connect them together in a modular, flexible and extensible manner.

For an arbitrary display wall application, one could imagine needing the following capabilities:

• A graphics view-port that is shown seamlessly across tiled displays
• 2D and 3D graphical elements within the view-port
• Computational functions and methods to alter the graphical elements or provide other functionality
• Manage assets, images and data and make them available to an application
• Provide user interaction via natural motion, 3D tracking, peripheral input or voice, and interface this with applications
• If more than one application is to be made available to the user, one would also require a method to manage and select applications

WallFrame at a Glance

Essentially, WallFrame is composed of a set of core functionalities, like user tracking, user management and gesture detection, , WallFrame Application plug into the WallFrame application manager and are accessible via the WallFrame App Menu. Other executables or programs in WallFrame will be referred to as nodes or modules

The Wall Framework

Below is a figure of what the different components are in Wallframe and how they fit together:

Components

Lets talk about the individual components shown above:

[The WallFrame Tracker](Wallframe Tracker)

* This is the interface to the middleware that talks to your tracker sensor (Kinect, Xtion) * Tracks users in the scene and publishes user position (joints) on the `/wallframe/tracker/users` topic * [More Information](Wallframe Tracker)

[The WallFrame User Manager](Wallframe User Manager)

* The user manager is where all of the magic happens for propagating user information to the rest of the WallFrame system. The information from `modulari_tracker` is fairly barebones, and also gives user position relative to the tracker device's coordinate system. * [More Information](Wallframe User)

Apps

• Apps in Wallframe are executables that are launched by the Application Manager as sub processes. This means that the app manager "owns" them, and can terminate them, when switching to another app.
• Each app needs to be wrapped by a ROS launch file.
• If the author so chooses, the app can provide interfaces (such as callbacks, subscribers, etc.) to recieve the messages sent by the Wallframe user manager. This is easily accomplished if the author uses the provide base classes, described [here](Base Classes and Sample Apps)

The App Manager

* This module loads a list of applications from the Application Manifest (a text file), then looks in a specified Application Directory for matching apps and their launch file.  Both the location of the app manifest and the application directory are set in the `wallframe_core.launch` file.  The app manager then sets a list of the available applications (apps that are listed in the manifest that are found in the directory) to the parameter server under both `/wallframe/core/available_apps` (for a list of all apps) and `/wallframe/core/available_app/app_name` (for an individual app).  These parameters contain the full path to an applications launch file. * The application manager knows where the launch file for each app is, and can launch that app as a child process, or close the app when prompted to do so via the following service calls: `/wallframe/app_manager/load_app`, `/wallframe/app_manager/close_app`, `/wallframe/app_manager/close_all_apps`

The App Menu

* The application menu is a user interface written in Qt. It uses params set in the wallframe_core.launch file to display the menu across the entire wall. * By using service calls, the app menu can signal the launching and closing of applications to the app manager. * The app menu shows icons and text pulled from each application in the app directory, and has no hard-coded knowledge of any of the apps. The app menu will automatically look at the available apps parameter set by the app manager, and load each app's icon and description text. The menu icons will scale based on the number of apps found in the app directory. * The user interacts with the menu by using his or her hand to control a cursor. When the cursor is over a desired app, the user then touches their elbow in a *click* gesture. This will be discussed in more detail below.

Next: [ROS and Wallframe](ROS and Wallframe)