By Lynn Thompson

Downloads

Download Unity* 3D Touch GUI Widgets [PDF 966KB]
Source Code: (coming soon!)

This article provides an example of using Unity* 3D assets to simulate Windows* graphical user interface (GUI) widgets. The TouchScript package available at no cost from the Unity Asset Store makes several touch gestures available. The example in this article uses the Press Gesture and Release Gesture.

The example begins with a preconfigured scene that has geometry for visual reference points and a capsule asset with a first-person shooter (FPS) controller and the scene’s main camera as a child for navigating the geometry. The scene is initially navigated in the default method, where an FPSController takes its input from the keyboard. To provide an alternate means of input for the FPS controller, I construct a GUI widget out of Unity 3D cubes. This widget will have four buttons: Up, Down, Left, and Right. I make the GUI widget visible on the screen by placing it in view of a second-scene camera with an altered normalized view port rectangle and a higher depth setting than the main camera. When these buttons receive a touch gesture, a respective action is sent to the FPS input controller to generate the desired effect.

I construct an additional four button widget to control rotation of the capsule asset that holds the scene’s main camera. These buttons enable users to manipulate the “first person’s” rotation independently of the FPSController, moving the asset at the same time. This functionality uses the Press and Release Gestures.

When complete, running this simulation allows users to touch multiple buttons in varying patterns to navigate the scene. After this, I explore possibilities for how you can use and modify the GUI widgets developed in this example to emulate other, common gaming controllers. I also discuss the challenges I experienced using touch-based controllers in contrast to keyboard and handheld controllers.

Creating the Example

I begin this example by importing a preconfigured scene I exported from Autodesk 3ds Max*, adding a capsule and configuring it with an FPSController. By default, this controller takes its input from the keyboard. See Figure 1.

Figure 1.Unity* 3D Editor with a scene imported from Autodesk 3ds Max*

Adding Geometry

Next, I add geometry (cubes MoveForward, MoveBackward, MoveLeft, and MoveRight) to simulate a Windows GUI widget. I also add a light and camera to visualize the newly added cubes. To place this camera’s view in the bottom left of the runtime scene, I change both of the normalized view port Rect settings for elements W and H from 0 to 0.25. Also, for the camera to appear in the scene, its Depth setting must be greater than that of the main camera. The Depth setting for the main camera is −1, so the default Depth setting of 0 for the new camera will work. I make the light and cubes children of the camera by dragging these elements onto the camera in the Hierarchy panel. Next, I add the TouchScript > Layers > Camera Layer to the camera by clicking Add Component in the Inspector panel. The GUI widgets won’t function if this Camera layer step is not performed. See Figure 2.

Figure 2.Unity* 3D Editor showing a GUI widget

Adding a GUI Widget

I repeat this process to add a GUI widget to the bottom right of the screen for rotation control of the capsule with the FPSController and main camera. The scene looks like Figure 3, with both of the GUI widgets added to the scene.

Figure 3.Unity* 3D runtime with imported scene and GUI widgets

Connect the Widgets to the FPS Controller

The next step is to connect the new GUIWidget cubes to FPSController. To do so, I modify the default FPS input controller script for the capsule to use variables to instigate movement rather than input from the keyboard. See script FPSInputController.js in the accompanying Unity 3D scene.

Adding Gestures to the Widget

Next, I add TouchScript Press and Release Gestures to each move GUIWidget cube by clicking Add Component in the Inspector panel for each cube. The TouchScript menu for selecting a gesture became available when I downloaded and installed the TouchScript package.

After the TouchScript has been added, I add a custom script to the cube to receive the gesture and perform the desired action. Choosing to start with CubeMoveLeft, I add a new MoveLeft script to the cube by clicking Add Component in the Inspector panel. This script sends a Horizontal value of −1 to the FPSController global variable horizontal when the cube receives a Press Gesture. I also add code to this script to change the scale of the cube to visually confirm receipt of the gesture. See the C# script MoveLeft.cs in the accompanying Unity 3D scene.

Similarly, I add scripts to send −1 to the MoveBackward button and 1 to the MoveForward and MoveRight GUIWidget cubes. See the C# scripts Move[Backward,Forward,Right].cs in the accompanying Unity 3D scene.

Enabling Button Functionality

At this point, I can use the Move GUI widgets to navigate the scene but only individually. I can’t use the MoveForward and MoveLeft or MoveRight buttons in combination to move at a 45‑degree angle. To enable this functionality, I create an empty GameObject at the top of the hierarchy and use the Add Component to add script Touch Manager from the TouchScript menu. I also add the script Win7TouchInput from the TouchScript Input menu.

Now that the Move buttons work and I can navigate the scene by touching multiple buttons, I’m ready to implement the rotation functionality. Theses buttons won’t manipulate the FPSController directly but the rotation of the capsule holding the FPSController and the scene’s main camera. Using the OnPress and OnRelease functionality as above, the script attached to the RotateLeft GUIWidget cube rotates the FPS capsule and child main camera to the left when the cube is touched. See the script RotateLeft.cs in the accompanying Unity 3D scene.

Similarly, I add scripts to send the appropriate rotation vector to the RotateUp, RotateRight, and Rotate Down GUIWidget cubes. See the scripts in the Rotate[Backward,Forward,Right].cs in the accompanying Unity 3D scene.

The Completed Example

This completes “hooking up” the cubes being used as GUI widgets. I can now navigate the scene with touch-controlled movement and rotation multiple ways by touching and releasing multiple buttons.

I added a script to the main camera to create a video of the scene being run. This script writes a new .png file each frame. See the script ScreenCapture.cs in the accompanying Unity 3D scene.

I compiled the .png files that this script writes into a video called Unity3dTouch2.wmv using Autodesk 3ds Max and Windows Live* Movie Maker. I removed this script from the main camera upon completion of the video because it noticeably degrades the performance of the scene when active.

Video 1: Touch Script Multi Gesture Example

Common Game Controllers

Common game controllers include first person, third person, driving, flying, and overhead. Let’s look at each.

First Person

When comparing the GUI widgets implemented in the example above to the stock Unity 3D first-person controller, one of the most noticeable differences is the GUI widget example getting the camera rotation in an odd configuration. When you use two buttons for capsule rotation, it’s not immediately obvious how to get the rotation back to the original state where the camera is in alignment with the scene horizon.

The stock Unity 3D first-person controller uses a script called MouseLook to perform the functionality that the Rotate[Left,Right,Up,Down] buttons provide. The MouseLook script uses localEulerAngles to rotate the camera; it offers a better means of rotating the camera view than the capsule rotation I used in the example. To take advantage of this better means of rotation, you can implement it in a manner similar to the FPSInputController: adding public variables mouseX and mouseY to the MouseLook script. You can then use these variables to replace the Input.GetAxis(“Mouse X”) and Input.GetAxis(“Mouse Y”) functions in the script. When these variables are hooked up to the rotate buttons and incremented and decremented, respectively, the scene’s main camera will rotate in a more useful manner.

Third Person

The stock Unity 3D third-person controller can be adapted to touch in a way similar to the first-person controller. Implement Move[Left,Right,Up,Down] in the ThirdPersonController.js script after hooking it up to the touch buttons with a new script, as before. The stock Unity 3D third-person controller automatically calculates the main camera rotation and position, leaving the second GUI widget created in the example available for alternate use. One possibility is to use the top and bottom buttons to increase and decrease variable jumpHeight, respectively, and use the left and right buttons to increase and decrease variable runSpeed, respectively. Many variables are available for similar adjustment in the ThirdPersonController.js script.

Driving

In the controllers examined so far, the Move[Left,Right,Forward,Reverse] scripts stop the motion of the character when an OnRelease event is detected. For a driving-type game, the scripts would do more than send a 1 or −1 to the first- or third-person controller. The Forward and Reverse scripts would have a range of values sent to emulate throttling and braking. The first 80% of the value range may occur rapidly when holding down the button for rapid acceleration; the last 20% of the values get slowly sent to the appropriate vector for slowly attaining maximum speed on a straight road while continually holding the Forward button down. The left and right buttons would perform similarly, possibly controlling the rotation of a Unity 3D asset that uses a wheel collider. In this type of scene, the GUI widget not used for steering can be used for control over parameters such as camera distance from the vehicle, throttle and braking sensitivity, and tire friction.

Flying

To use the GUI widget interface developed in the example for a flying-type game, you would use the Move[Left,Right,Forward,Reverse] buttons similar to a joy stick or flight stick. The left and right buttons would adjust roll, and the up and down buttons would control pitch. The Rotate[Left,Right] buttons in the other GUI widget can be used to increase and decrease yaw and camera distance from the aircraft.

Overhead View

In this type of scene, the main camera orbits the scene from overhead, likely moving around the perimeter of the scene while “looking” at the center of the scene. In a script attached to a Unity 3D scene’s main camera, you could define several Vector3s at points along the perimeter of the scene. Using the Vector3.Lerp function, you can control the fraction parameter with the MoveLeft and MoveRight GUI widget buttons to move the camera between two of the perimeter points. The script can detect when a perimeter point has been reached and begin “Lerp’ing” between the next two Vector3 points. The MoveForward and MoveReverse buttons can be used to adjust the vertical component of the Vector3 points to move the orbiting camera closer to or farther away from the scene. You could employ the other GUI widget being used for Rotate[Left,Right,Up,Down] in the example for a wide variety of things, such as time-of-day control or season-of-year control.

Issues with Touch Control

The most readily observed issue in using touch control in the example above is that it blocks the view of the scene in the lower left and lower right corners. You may be able to partially remedy this problem by getting rid of the cameras that view the GUI widget buttons and make the buttons children of the scene’s main camera. The buttons would still be in the scene but would not be blocking out an entire rectangle of the scene.

You could further minimize button visibility by making them larger for more intuitive contact, and then making them disappear when touched and reappear when released. You can achieve this disappearing and reappearing by manipulating the asset’s MeshRenderer in the onPress and onRelease functions as follows:

GetComponent(MeshRenderer).enabled = false;
.
.
.
GetComponent(MeshRenderer).enabled = true;

Another challenge of using touch is ergonomics. When using touch, users can’t rest their wrists on a keyboard and may not be able to rest their elbows on a desk. When developing GUI widgets for touch, take care to place buttons in the best position possible and use of the most efficient gesture possible.

Conclusion

The TouchScript package functions well when implementing the Press and Release Gestures. The resulting Unity 3D scene performed as desired when developed and run on Windows 8, even though the TouchScript input was defined for Windows 7.

The more common gaming interfaces can be emulated with touch. Because you can implement many combinations of touch gestures, many options are available when implementing and expanding these emulations. Keeping ergonomics in mind while implementing these GUI widget interfaces will lead to a better user experience.

About the author

Lynn Thompson is an IT professional with more than 20 years of experience in business and industrial computing environments. His earliest experience is using CAD to modify and create control system drawings during a control system upgrade at a power utility. During this time, Lynn received his B.S. degree in Electrical Engineering from the University of Nebraska, Lincoln. He went on to work as a systems administrator at an IT integrator during the dot com boom. This work focused primarily on operating system, database, and application administration on a wide variety of platforms. After the dot com bust, he worked on a range of projects as an IT consultant for companies in the garment, oil and gas, and defense industries. Now, Lynn has come full circle and works as an engineer at a power utility. Lynn has since earned a Masters of Engineering degree with a concentration in Engineering Management, also from the University of Nebraska, Lincoln.

Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.
Copyright © 2013 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.

 Que educação e jogos estão cada vez mais ligadas, não há duvidas, ainda mais quando o jogo é em RA!

 O jogo é uma ótima ferramenta de aprendizagem, na medida em que propõe estímulo e interesse ao aluno. Porém, recursos de jogos são limitados devido à complexidade imposta por determinadas tarefas. Jogos educacionais desenvolvidos em ambiente de Realidade Aumentada possibilitam um maior realismo e interatividade, visando à melhoria do processo de aprendizagem do educando. A tecnologia de RA em atividades lúdicas é muito poderosa, podendo proporcionar maior envolvimento e motivação aos alunos. Jogos que envolvam gincanas e buscas em espaços abertos são bastante eficazes.

Depois de analisarmos como insuficientes as ações em Pernambuco que exploram a capacidade sociocultural no que tange as produções digitais de advergames e programas educacionais modernos em formatos de jogos e aplicativos para computador, smartphone, tablets, etc  propusemos alternativas pra suprir essa necessidade.

Depois de várias pesquisas sobre o déficit em que o pais se encontra com a educação e de muitas pesquisas foi criado o EducAR!  

MAIS SOBRE O PROJETO:

O projeto trás muita inovação e interação para os usuários! E foi um sucesso ainda em sua fase de testes na versão mobile!

O EducAR se trata de um app onde a intenção é fazer com que os alunos possam aprofundar, disseminar e interagir em um ambiente digital, coletivo e gamificado onde aborda o assunto que foi ou irá ser tratado em sala de aula, para despertar o interesse e reforçar o aprendizado.
O App é atualmente desenvolvido para a plataforma Android.

INTEL PERCEPTUAL CHALLENGE: 

 Após tomar conhecimento sobre o concurso, como somos apaixonados por tecnologia e estamos sempre querendo inovar, decidimos tentar adaptar o projeto para a Perceptual Câmera. 

No começo, propomos na submissão do projeto, desenvolver só a tecnologia de Realidade Aumentada e caso conseguíssemos em tempo hábil, tentaríamos implementar os movimentos para interação com a Realidade Aumentada, mas eis que surgiram vários problemas!

DESAFIO DA ADAPTAÇÃO DO PROJETO: 

Como o projeto é desenvolvido para dispositivos mobile, uma grande dificuldade que encontramos foi arrumar uma forma de conciliarmos o uso do marcador com os movimentos, de uma forma que uma coisa não passe na frente da outra para não interromper o tracker do marcador. Tentamos usa-lo numa mesa, mas ficaria ruim de executar os movimentos na câmera. Com esse empecilho desenvolvemos uma espécie de "babador" como marcador, onde assim poderíamos projetar a RA e ao mesmo tempo usar os movimentos de uma forma confortável!

(Não conseguimos mudar a imagem do marcador na época, mas hoje o problema já está sendo corrigido)

Dessa forma desenvolvemos o EducAR Intel Perceptual Version (demo), onde o apoio e suporte da Intel foram de extrema importância para conseguirmos desenvolver tal app! O fórum nos ajudou bastante!  

Aqui um teaser do EducAR Perceptual Version.

 No concurso havia alguns projetos com a tecnologia de RA, mas como o SDK ainda estava em desenvolvimento estava um pouco instável, onde conseguimos contornar alguns problemas. (Alguns bugs já foram resolvidos no novo release do SDK 5). 

Agora tentaremos implementar o comando de voz, pois quando tentamos na versão anterior nem o nosso OK pegava direito, deve ter sido pelo sotaque nordestino de ser. hahaha :)

MAIS SOBRE A EMPRESA PRODUTORA:

Fundada em 2013, a RApp´s Studio é uma Start-Up Pernambucana, encubada no Instituto de Tecnologia de Pernambuco (ITEP), trabalha projetando e desenvolvendo soluções e conteúdos digitais na area de TI, com a tecnologia de Realidade Aumentada, como advergames, aplicativos e peças publicitárias em geral. Empresa essa que utiliza o enfoque da tecnologia para proporcionar e destacar o fluxo de imersão nos produtos, oferecendo uma poderosa relação entre cliente/produto/serviço.

LINKS: 

- Para conhecer um pouco mais sobre a empresa produtora do App acesse: http://www.rapsstudio.com

A equipe RApp´s Studio agradece a Intel por todo apoio e suporte!

For 24 hours in early November, we held the 2nd GA Tech Code for Good Student Hackathon.  In continuation of last year's event here, we retained the theme of teaching healthy lifestyle choices to combat childhood obesity.  From edutainment to exercise games, we seek to create worthwhile projects that can help an at-risk demographic: our future.

With Intel providing the food and Android tablets, the students have been working non-stop on these beneficial games.  Our host at Georgia Tech is Professor Matthew Wolf.  Special guest Cornelia Davis from Pivotal labs joined us to share her expertise on Cloud Foundry, with which the students have hosted and distributed their software. 

Variations on the Theme

From the previous hackathon on this subject, domain experts share insights:

From Healthier Generations -

 Are there technologies that solve similar problems?

Perhaps you’re inspired by a feature of another piece of technology such as an app on your phone, or an online service. Do you know of other technologies that solve similar problems, or solve a problem in a similar way to what you imagine?

Two apps that do some of the things that we think are important are Instagram* and WebMD*. 

Instagram - people can take photos, put them on a map and connect with others through images. In case of childhood obesity, they could take photos and/or map comments about their environment as it relates to access to healthy food and safe places for physical activity. 

WebMD* - similar to how WebMD identifies symptoms and treatments, we would like to offer questions about a person’s environment and help them identify solutions in their environment.

From Dr. Marks

1) the most important thing is to get people moving.  Hopefully walking, but at least moving.  Games that require and reward the kids to actually walk to move the character through the game would be great.

2) Nutrition that not only rates meals, but also allows them to have nutrition information in an understandable format, relevant to school lunches, would also be good.  The overwhelming majority of school foods in this country are provided by a single company, so this is do-able.  It does need to be fun, or kids won't do it.  You can also take advantage of the cameras that most cell phones have these days.  Is there any way to photograph a school lunch and cross reference it with the known inventory of the company supplying the food?  Could you have some kind of reference item of known shape and size that gets photographed with the food so that portion sizes can be estimated?

3) knowledge is power.  Kids that know where their food came from make better choices.  How many kids know, for example, that ketchup is mostly high fructose corn syrup?  Do they even know what a tomato is?

4) kids do in fact educate and pressure their parents in very meaningful ways. The question is how to build in motivation and reward on both sides.

5) is there any way to turn a standard phone into a pedometer?  Can you track how much a child moved so that appropriate rewards can be offered?

6) improving our ability to move through the built environment is key.

There are many map programs that calculate driving routes. Is it possible to calculate the best/safest walking or biking route?

The Teams

Team: "MEM@" 

Game: Geocaching mystery game, find virtual clues at physical locations.  Encourages physical activity by movement among target places.

The team started work using the Cloud9 IDE, an online collaboration tool, while investigating geolocation, Google Maps API, and MongoDB for use with Cloud Foundry.  Unfortunately their lead coder disappeared for a few hours, so the team iterated on high-level design and architecture.  Upon his return, they consolidated their work and reached a working prototype.  Despite trouble integrating with the server, this team's demo was functional by the end of the event.

Final report out: http://www.youtube.com/watch?v=iEz3q_ibC04

Team: RADD

Game: Multi-device whack-a-mole style monster catching game.  Requires physical activity to catch monsters, thus gaining points.

From the start, the team split into pairs working on the frontend and backend, using the Handlebars.js templating engine and MongoDB respectively. The interfaces were completed quickly, followed by the registration/login features, but the team hit a snag with MongoDB authentication issues.  As soon as the problems were ironed out, integration went smoothly.

 The final demo was as entertaining to watch as it was to play: http://www.youtube.com/watch?v=pTsOadJAsn4

Final report out: http://www.youtube.com/watch?v=TWbOsAFF8kg

Wrap-up

By the end of this event, all involved had enjoyed the time but were ready to rest.  The timing was not ideal- being right after GA Tech's homecoming, when pre-Thanksgiving projects are all coming due- and there was another event happening right down the street, but our reduced turnout allowed closer work with the teams.  This also resulted in the highest percentage of demo-ready apps per team, as both teams reached that milestone.  I look forward to working with GA Tech again next year.

Links

Hackathon announcement: http://cercs-ed.gatech.edu/activities/2013hackathon

GA Tech CERCS live blog: http://cercs-ed.gatech.edu/activities/2013hackathon-liveblog

Cornelia's Cloud Foundry blog coverage: http://blog.cloudfoundry.com/2013/11/14/georgia-tech-hack-for-good-on-cloud-foundry/

The Intel Software and Services Group (SSG) opened its first office in East Africa in April 2013. This was a big move for Intel, which had previously only had offices in South Africa and Egypt. The local SSG is responsible for working with East African developers to provide them with design tools, resources and expert consulting. 

In a bid to find out more about what it means for local Kenyan developers to have the SSG available to them, we chat with Fredrick Odhiambo, an application engineer with Intel East Africa. A graduate of the University of Nairobi, he focuses on the developer space and seeks to enhance local innovation and provide technical expertise support and tools to developers that enable them to create applications with rich user experience on devices running on Intel technology.

Q: Why does Intel care about software? As far as most people/developers are concerned Intel deals solely with hardware - how do you make the connection with software?

A: The Software and Services group works closely with independent software vendors and operating system vendors. Our aim is to enable them to make the most of our hardware by developing software that runs optimally on our platform. 

Q: Why should I care about going native as a developer? Android is Android. What does going native even mean? 

A: Going native means developing differentiated Android apps using native programming languages like C, C++ or Assembly Language. Native development gives the developer a better, differentiated app that takes advantage of direct CPU and hardware access. Native development is good for performance intensive tasks like signal processing, image manipulation and complex algorithms.

As for numbers from Google Play, 68% of the top 300 free apps on the Play store are native Android apps, and only 32% are Dalvik Android apps. This trend is the same for the top 300 paid Android apps, where 80% are native Android apps. 

Q: How many apps have you developed so far? 

A: 5 apps, all on different platforms i.e. J2me, Android, Qt and HTML5.  

Q: What has been your most successful app? Why? 

A: My most successful application was a mobile phone utility for the blind, a free text to speech application based on low end, inexpensive java phones. The application helps visually impaired persons to use their mobile phone utilities with ease, i.e. calculator, SMS, phonebook etc. The application is in Swahili which is the most widely spoken language in East Africa, after English. The app is being used by visually impaired and blind people across the country and has had a huge impact on the lifestyle of the disabled. 

Q: You have been seen pushing for the Havok gaming engine. Why would a game developer pay attention to this particular gaming engine?

A: Havok enables developers to create world class gaming apps, and it’s totally free. The cross platform supports Android, iOS and Tizen; and has a published roadmap with regular releases. We are currently running an 8 week fully sponsored training program in Nairobi on how to build apps on the Havok gaming engine.

Q: What are you learning right now? 

A: Perceptual computing. Perceptual computing focuses on research and development of technologies that leverage on natural user interactivity enabling users to interact with their PCs in more intuitive, natural engaging ways. Main modes of interaction are speech recognition, finger/hand gestures, face recognition and augmented reality. A typical example is the facial login, no passwords required at all, the system “remembers” the user. 

Q: What is important for developers in Africa to know regarding how they can benefit from Intel? 

A: Sign up at Intel Developer Zone Africa. This is a global program that enables developers to engage with Intel on topics related to software and is designed to answer today’s software development challenges. The portal gives developers to our latest software development and optimization tools. 

Q: Thanks for your time Fred! Any parting shot? 

A: Let’s go native! 

Download Article

Not Built in a Day – Lessons Learned on Total War: Rome* II [PDF 935KB]

Abstract

The developers at Creative Assembly had a challenge: How could they get Total War: ROME* II to play well across a wide range of Intel® systems without compromising the game aesthetics? This case study details how the game takes best advantage of low-power systems, while still scaling up to look and run fantastic on more robust systems.

High-fidelity landscapes are an essential part of the game’s rich historical environments, and lush foliage is key to those landscapes. The foliage was optimized with Adaptive Order Independent Transparency (AOIT), giving it a rich look with low performance overhead. The team added a game benchmark for an easy way to measure performance. An in-game battery meter allows players to monitor power when playing on the go. They also tuned for several different systems at once to ensure that any optimizations were balanced across systems, and added detection code to automatically set the right options for each system.

The team made improvements to many other areas, including LODs, shadows, landscape generation, particles, CPU tasks, and sound. They also optimized for memory bandwidth.

Together, these gave the game great performance across a wide range of Intel systems.

The challenge

When building Total War: ROME II, Creative Assembly challenged Intel. They wanted to deliver the most immersive experience of any Total War game to date, on all types of systems, with no compromise. But today’s systems have a variety of features and come in a number of form factors. How could they deliver great gaming at fluid frame rates across all of these systems? They turned to Intel’s engineers. Together, we delivered a fantastic game that players can enjoy across the full range of the latest Intel systems, from the most power-thrifty Ultrabook™ up through full power laptop, all-in-one, and desktop systems.

Figure 1. Typical scene

The team put their requirements on a sliding scale, with more capable machines delivering an even faster frame rate with higher resolution and quality settings. Since the game is faster on more capable systems, advanced features are enabled so the game renders with even higher quality. This includes AOIT built on top of the Intel® Iris™ graphics extension for pixel synchronization, which gives systems with Intel graphics much faster transparency calculations.

To make sure that the game is playable across a wide range of systems, it automatically configures itself to match each system. When the system is running on battery, the game displays a battery meter. ROME II also includes a built-in benchmark mode so anyone can see the game’s performance.

The team studied the GPU and CPU performance of the game with Intel® Graphics Performance Analyzers (Intel® GPA), Microsoft GPUView, and Intel® VTune™ Amplifier XE.

As we walk through this case study, you should see similarities with your game development. We hope this case study helps you implement similar features in your game.

Detecting the platform

With code like the GPU Detect code sample, ROME II detects the system’s graphics device. With this information, the game configures itself for the best visual fidelity on each system. On systems with Intel® HD Graphics 4200/4400/4600 (typically systems that use 15W of power), the game defaults to 1366x768 resolution and Medium quality.

Figure 2. Sample scene on Intel® HD Graphics 4600 system

For systems with Intel HD Graphics 5000 and Iris Graphics 5100, the game sets itself to 1600x900 with Medium quality, with increased shadow fidelity.

On systems with Iris™ Pro Graphics 5200, the game defaults to 1600x900 with High quality, using AOIT for even better visual quality.

Figure 3. Sample scene on Iris™ Pro Graphics 5200

Extensive benchmarking proves that the game has great frame rates (>=30 FPS for at least 95% of typical game play) on all these configurations.

Setting the bar with a benchmark

To make it easier to measure the game’s performance across a variety of systems, ROME II includes a benchmark to showcase typical performance across a campaign scenario. Go to the advanced graphics options screen and select “run benchmark”.  For simpler benchmarking, it can also be started from the command line. Although the game ships with a single benchmark, it has a benchmark selection screen, so it can use additional benchmarks.

The benchmark doesn’t specifically measure power. If you want to study power during gameplay, run the benchmark and a power-monitoring tool (see Intel® Graphics Performance Analyzers (Intel® GPA) System Analyzer for real-time power measurement).

We recommend you build a benchmark like this (plus a benchmark-running tool), to show your game’s typical performance.

We’ve got the power

To showcase how long the game can last on battery, it includes a battery meter. The meter is hidden if the battery is fully charged and plugged in to AC power. Anytime an Ultrabook or laptop system is running on battery or charging from AC power, the battery meter appears so the player knows how long they can play. The battery meter has been integrated into the screen so that it doesn’t hide any essential information.

Figure 4. Battery meter at the top of the screen

Some games adapt to battery power by reducing resolution and quality settings, or reducing frame rate. These strategies did not give a satisfying gaming experience in ROME II, so the team did not include any specific power optimizations.

As you work on your game, study how it’s using power, and see if some of those optimizations might be right for you.

AOIT makes the vegetation look good

The Total War games are known for their immersive environments, with realistic foliage. This aesthetic requires transparency, and lots of it.

For the game to look its best on Intel graphics hardware, it uses an Intel Iris graphics extension to the DirectX* API. The pixel synchronization extension gives a low-overhead way to synchronize pixel writes via the graphics driver, which accelerates transparency.

The game originally used an Alpha-to-Coverage solution for transparency. The team planned to supplement this with a k-buffer solution in ROME II. They found that the k-buffer works fine for small areas of the screen with a fixed amount of transparency. However, there were problems on a full screen with the levels of overdraw seen in ROME II. It quickly ran out of GPU memory, so it ran far too slowly. AOIT doesn’t suffer from that problem, and is about 5x faster than k-buffer. AOIT also lets the player see more alpha around the edges of the leaves. This gives a better appearance of depth, especially at resolutions well below 1080p.

While a general AOIT algorithm was published a few years ago, a recent code sample details how to accelerate AOIT with pixel synchronization. With pixel synchronization, shaders write colour and depth into an Unordered Access View (UAV) buffer. The farthest colours are blended as they are written. Then, a visibility function (VF) combines the colours with a refactored alpha-blending equation. Together, this gives a deterministic and fast way to figure transparency.

The AOIT pixel synchronization sample was literally “dropped in” to the ROME II code base with few changes. The ROME II version of AOIT has a pre-multiplied alpha and does some culling. It was easy to add lighting to the AOIT pass and let the tree foliage cast its own shadows.

For higher-end systems, AOIT is now the default for transparency. All other configurations use Alpha-to-Coverage.

Figure 5. Vegetation looks great, thanks to AOIT

AOIT operates on the foreground and middle distance vegetation, with great results. While it’s made this screen shot look great, the actual movement in the game is even better. AOIT should be just as easy to include in your game.

Bandwidth optimizations

After studying the game, we felt that reducing bandwidth at the GPU would increase overall performance. With multiple render targets, the game wrote to a single output format, all the same size. Since it’s possible to have different output formats and bit widths on each render target, we changed the game so it picks appropriate formats and sizes for each.

The landscape engine generates textures. It stores surface colour and normal as RGBA 8 textures, even though the alpha channel is unused. A more efficient format can reduce the bandwidth.

The terrain sampler used manual bilinear filtering. The team studied the vertex shader’s profile in Intel GPA Frame Analyzer, and found the shader was too slow. It read 4 heights with a gather4, and then performed a manual bilinear filter of the 4 values. The vertex shader now has a sampler with appropriate filtering, so it’s quicker.

Figure 6. Profiled times for manual bilinear filtering in the terrain vertex shader

We discovered some LOD models were not optimal. For example, the torso animation model is instanced throughout a campaigning army. But Intel GPA revealed those vertex shaders were invoked too often. This is a common issue so we usually check for it. Instancing can be effective when rendering a whole group of objects with the same model using the same mesh. However, when the objects are dispersed throughout the scene with very different Z depths, there are often a large number of objects in the distance. They render to very small parts of the screen, giving lots of sub-pixel polygons. Even though the instancing reduces the number of draw calls, the number of vertex shader calls on those sub-pixel polygons can become very inefficient.

To avoid this, be cautious with Z depth when instancing. For complex models, only instance them if the Z depth of the instance is a fairly close match with the Z depth of the original.

Figure 7. Vertex shader invoked as often as pixel shader, a clear sign of trouble on this instanced mesh

In general, the vertex shader should be invoked much less often than the pixel shader, and any large number of primitives, post-filter texels and reads may indicate the same issue. In this example, the vertex shader was invoked as often as the pixel shader. To fix this, the game switched to a simpler LOD model for distant objects.

Shadows

Shadows revealed a number of issues that were all ultimately improved.

We had an issue with the number of cascades. ROME II had a cascaded shadow map, which allows great detail in close up areas but still maintains a wide area of shadow detail. At first, the game had three cascades that needed depth tuning for the best effect. By carefully placing the division between cascades, we reduced this to two cascades. The shadow effect was still good, but the game ran faster.

Shadow generation and the main rendering pass both had the same vertex signature (11 inputs). This was inefficient since shadow generation didn’t need a large part of that vertex signature. We couldn’t simplify the vertex signature, however. Many components of the scene need alpha or punch-through areas of their textures, so texture processing was necessary for shadow creation. Later in the project, the input signatures were separately reduced (by 3 float4s), giving shadows a small performance gain.

Shadow maps suffered from the same sub-pixel geometry issue created by distant instanced objects. When the LODs of the main scene were tuned (see above), the shadow map creation improved similarly. The game got a small speedup by changing to LOD models, with impostors for distant models.

Landscape

Originally, the landscape was tessellated in screen space. This resulted in high polygon counts, so we replaced it with a tiled renderer. This had a high vertex shader cost, but rendered much faster (from 5.4 ms to 2.4 ms on the same scene).

ROME II uses very large landscape textures. The visible area is generated in real time. Height and terrain information is composited on the GPU and stored in a texture atlas, but required careful tuning. Each frame renders enough tiles to display newly visible areas, while limiting its work so that it doesn’t take too much GPU and interfere with the rest of rendering.

Particles

Looking more closely at a typical frame capture, there was an expensive section of work that had little effect on the frame.

Figure 8. Particles were expensive but had little effect

This was due to particles, which were each drawn as a separate polygon. They had no impact on the scene, so they could be removed for a large speedup.

Although it seemed that particles might not interact well with AOIT, the new particle engine developed for the game worked well and had no problems with AOIT.

Tuning the CPU-side code

Although graphics received a lot of attention during this project, the team also wanted to optimize the CPU side of ROME II. With Intel® VTune™ Amplifier XE, the team studied the game for CPU bottlenecks, and several areas yielded impressive gains.

The sound engine took more CPU time than expected. While it’s a powerful sound engine, capable of sophisticated mixing and blending on the CPU, it ran at its highest detail level even when set to “normal.” Fixing this sped up frames by up to 1.1x. Lower-power systems benefited from this, yielding a better frame rate and longer battery life.

The game includes a task-based threading system. The tasks vary in size, and some of them didn’t interact well with the automatic task scheduling from the task pool. This caused “bubbles” in the schedule and slowed the frame. Problematic tasks were removed from the task list and manually scheduled on their own thread, separate from everything else. This gave an optimal thread schedule.

Conclusion

Working together, Creative Assembly and Intel carefully studied ROME II during development. Using Intel GPA, Intel VTune Amplifier, GPUView and deep analysis, the team found many issues to improve. Together, the team tuned many parts of the game and integrated industry-leading algorithms.

Since the game automatically configures itself for each system, it’s faster in many cases than it would have been. The battery meter and in-game benchmark make it possible to study the game during gameplay or for benchmarking.

Foliage benefited from AOIT, LODs were isolated to the right depth and replaced with impostors in the distance, shadows are much faster, the landscape and particle systems work better and faster, and the sound and task systems use the right workloads at the right times.

Together, this all lets Total War: ROME II look and run great on Intel platforms. We hope this case study helps you do the same with your next game! Let us know what you think.

Huge thanks to Creative Assembly for building a great franchise and extending ROME II to shine on Intel platforms. Special thanks also to Steve Hughes of Intel, who has worked with Creative Assembly on a number of Total War games, delivering the best results to date in ROME II.

About the Author

Paul Lindberg is a Senior Software Engineer in Developer Relations at Intel. He helps game developers all over the world to ship kick-ass games and other apps that shine on Intel platforms.

Intel, the Intel logo, Iris, Ultrabook, and VTune are trademarks of Intel Corporation in the U.S. and/or other countries.
Copyright © 2013 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.

These events can show non-coders new potential, teach effective software development practices, help students acquire specific technology and interpersonal skills, and bridge the gap between academia and the real world.  In school, you learn and then apply; in the real world, you have to apply without learning.  These events help participants “learn how to learn,” learning through application.

The student-led hackathons will generally be remotely supported by Intel (funding, video calls, target platforms, etc.), run by student ambassadors with guidance and facility management from experienced faculty.  One (or more) of the planners should be in charge of maintaining an event blog and gathering the code (generally on GitHub) for posterity.

Once your core team is in place, it’s time to start planning.

Preparation

• Theme
  • “Code for Good”- beneficial to society (e.g. teaching middle school students basic algebra, healthy lifestyle choices to combat childhood obesity, etc.)
  • Not so general as to give no guidance or ideas
  • Not so narrow as to specify app to be created
  • Local community concerns are a good place to start
• Participants
  • Early signup
  • Commitment
  • Ongoing communication to maintain preparation and involvement
• Internet
  • Ethernet connections as backup
  • Power outlets and strips
  • Any platforms and servers required
• Food
  • Meals
  • Grazing between
  • Caffeine for late night boosts
  • With Intel buying the food, it’s often best to order online (from Safeway or similar)
• Facilities
  • Power
  • Lights
  • AC
  • Unlocked door(s) for access
  • Parking and security if necessary
  • All of the above assured overnight
• Swag
  • T-Shirts
  • Stickers
  • Prizes
  • Certificates

Ordering t-shirts

A few things to keep in mind to keep the shirt designs standardized and useful:

• Color limitation (not counting shirt color)
• Badge section on front
• Full event design on back
• Code for Good logo on right sleeve
• Mix of sizes or ask during signup

Specific technology training

Some tools, engines, and technologies have a steeper learning curve than others.  Make sure to prepare a workshop before the hackathon when using these or the work time will be drastically inhibited by learning how to use them.

Downloadable toolkit image

It’s often helpful to package up the relevant tools for quick deployment to all development platforms (often student laptops).  Here’s an example bundle:

• Notepad++ for basic text editing
• GIMP for images, Aseprite for sprites
• Possibly a level editor such as Ogmo, Tiled, or Dame
• Audacity, bfxr and/or musagi for audio
• Event-specific tools such as XDK or Project Anarchy

Remember to consider the mix of operating systems.

Tips and tricks for a successful hackathon

• Games keep participants motivated and are easier to demo/see progress
• Competitive events would only appeal to half of the participants; collaboration is better for learning
• Rather than judging with rankings, give tickets when participants exhibit beneficial behaviors or reach milestones.  At the end, raffle prizes small to large.
• Commitments should be for the entire duration; having a freeform “drop in and out” arrangement might feel more accommodating, but the damage to the team dynamic severely impedes productivity.
• Schedule beforehand to have status check-in times at regular intervals (1-2 hrs).
• Schedule meal breaks to not be near or after check-ins.  Make sure these are at standard times- hungry people don’t care about much but food.
• Maintain decorum during check-ins.  Working through someone else’s status is fine, but having a disruptive conversation is simply rude.
• Check in the loud groups first to quiet their conversations.  If all are quiet, go with the quietest to stir them up.
• Predetermine milestones.  Estimate the first 25% for design and prototyping, alpha (functional engine working) by 50%, beta by 75% and gold by 90% to leave time for demo/wrap up.
• “However long you think something will take, triple it.” – Jon Shafer. By that same note, cut your dev time into a third; if you’re doing a 24 hours hackathon, what can you get done in 8 hours?  Not a sprawling MMO, aim for smaller scale.
• Programmer art is fine.  If you have a fun game that stars only boxes, you have a fun game.  Polish is for later.
• Have a camera, take pictures.  Prep time, work time, check-in time, wrap-up, all of these are ripe with photo ops.
• Similarly, have a video camera.  Record check-ins, demos, and really any time someone talks to the group.  It’s interesting to see the evolution of the games and teams over the course of the event.

More basic DIY hackathon tips are available at http://software.intel.com/codeforgood/hackathon-in-a-box/

Download Article

Download Intro to Motion Estimation Extension for OpenCL* [PDF 660KB]

This article introduces Intel’s motion estimation extension for OpenCL*. This extension includes a set of host-callable functions for frame-based Video Motion Estimation (VME).

This extension depends on the OpenCL 1.2 notion of the built-in kernels and on the cl_intel_accelerator vendor extension, which provide an abstraction for the specific hardware-accelerated capabilities.

This article provides a brief overview of the cl_intel_accelerator and cl_intel_motion_estimation extensions. A code example using these extensions is also included along with an explanation of its results.

For more information on extensions, refer to the cl_intel_accelerator and cl_intel_motion_estimation extension descriptions at the Khronos API registry.

Motion Estimation Overview

Motion estimation is the process of determining motion vectors that describe the transformation from one 2D image to another, usually from adjacent frames in a video sequence. The motion estimation functions, considered in this article, accept full-frame single-channel (luma) images as input, perform a motion search operation, and return a motion vector field as output.

The introduced VME functionality exposes part of the hardware acceleration pipeline for video acceleration. This VME extension provides low-level functionality, currently restricted to the single-channel (luma) input images and block matching methods, so motion vectors are computed for rectangular pixel blocks. Motion vectors are key elements in the video compression algorithms.

Motion vectors are useful for several applications. For example, when generating “slow motion effects,” motion vectors can provide the basis to generate intermediate frames for frame rate (up)conversion. Another example is increasing the original frame rate of the digitized film (24 fps) to match the TV rate.

Motion vectors are also useful for image stabilization: the motion vectors in the entire frame can be averaged to produce a “global” motion vector that can serve as an approximation to a real video camera motion.

The motion estimation extension consists of the new OpenCL built-in kernel (see section 5.6.1 in the OpenCL 1.2 specification) which performs motion estimation, as well as the accelerator object, which represents the state of the underlying acceleration engine. The kernel is queued for execution from the host using the standard ND-range mechanism.

Both cl_intel_accelerator and cl_intel_motion_estimation extensions should be listed in the CL_DEVICE_EXTENSIONS string (see Table 4.3 in the OpenCL 1.2 specification) for the Intel® HD Graphics device in your system. Otherwise you need to update your GPU driver first.

General Accelerator API

Creating an accelerator object

Accelerator objects provide a black-box abstraction of software- and/or hardware-accelerated functionality from OpenCL vendors. Intel cl_intel_accelerator vendor extension consists of a unified set of OpenCL runtime APIs to create, query, and manage the lifetime of the accelerator objects. The interfaces for this extension are provided in the cl_ext.h header.

Just as with other vendor extension APIs, the clGetExtensionFunctionAddressForPlatform function should be used to get pointers to the accelerator APIs:

static clCreateAcceleratorINTEL_fn pfn_clCreateAcceleratorINTEL = (clCreateAcceleratorINTEL_fn)
clGetExtensionFunctionAddressForPlatform(intel_platform_id, "clCreateAcceleratorINTEL");

clCreateAcceleratorINTEL_fn is defined as an appropriate function pointer in the cl_ext.h.

Accelerator object instances are referenced with the generic cl_accelerator_intel type. Notice that every accelerator is always associated with a specific acceleration engine type, which is requested by the application at accelerator object creation time. In the example below, the accelerator type is CL_ACCELERATOR_TYPE_MOTION_ESTIMATION_INTEL. Also, descriptors are used to request acceleration engine-specific properties:

cl_motion_estimation_desc_intel desc = {
CL_ME_MB_TYPE_16x16_INTEL,                                     
CL_ME_SUBPIXEL_MODE_INTEGER_INTEL,              
CL_ME_SAD_ADJUST_MODE_NONE_INTEL,                 
CL_ME_SEARCH_PATH_RADIUS_16_12_INTEL              
};
cl_accelerator_intel accelerator = pfn_clCreateAcceleratorINTEL(context, CL_ACCELERATOR_TYPE_MOTION_ESTIMATION_INTEL,
        sizeof(cl_motion_estimation_desc_intel), &desc, &err);

Refer to the full motion estimation extension specification for the descriptor details. Make sure to handle potential failure of the creation routine when clCreateAcceleratorINTEL returns zero for the accelerator handle value. Possible reasons for accelerator creation failure are invalid descriptors or an invalid combination of descriptor values. The extension specification lists all possible error codes and causes.

clReleaseAcceleratorINTEL is a complement to the creation API we just discussed. Refer to the Full Frame Motion Estimation Code Example section of this article for the example code.

Using the accelerator object

An application can run the accelerated motion estimation functions on an OpenCL device by enqueuing one of the proposed built-in kernels (below). The kernels are enqueued for execution by the regular clEnqueueNDRangeKernel OpenCL routine. In turn, a motion estimation accelerator encapsulates the internal state of the motion estimation engine and serves as the kernel argument to the motion estimation built-in kernel. The relationships between the entities are outlined in the following diagram:

Motion Estimation API

Notion of built-in kernels

Section 5.6.1 of the OpenCL 1.2 specification introduces the notion of built-in kernels. More specifically, clCreateProgramWithBuiltInKernels creates a program object given a context and loads the information related to the built-in kernels into the program object. Notice that the developer does not provide program source code for built-in kernels.

cl_program program = clCreateProgramWithBuiltInKernels(context,1,device,"block_motion_estimate_intel",&err);

The specific built-in kernels are created from the resulting program object:

cl_kernel kernel = clCreateKernel(program, "block_motion_estimate_intel", &err);

The kernels can be enqueued for execution by the OpenCL runtime using clEnqueueNDRangeKernel.

Built-in kernel for the motion estimation

The cl_intel_motion_estimation extension introduces a new built-in kernel for motion estimation with the following signature:

_kernel void 
block_motion_estimate_intel
(
accelerator_intel_t accelerator,
__read_only  image2d_t src_image,
__read_only  image2d_t ref_image,
__global short2 * prediction_motion_vector_buffer,
__global short2 * motion_vector_buffer,
__global ushort * residuals
);

This kernel computes motion vectors by comparing a 2D image source with a 2D reference image, producing a vector field of motion vectors. The algorithm searches the best match of each pixel block in the source image by searching an image region in the reference image, centered on the coordinates of that pixel block in the source image (optionally offset by the prediction motion vectors).

When enqueuing this kernel, global_work_size and global_work_offset determine the region of interest of the input frames. The dimension of the output motion vector image is dependent on the size of the region of interest and partitioning mode specified by the accelerator.

accelerator should be a valid accelerator object created by clCreateAcceleratorINTEL, where the type of the accelerator must be CL_ACCELERATOR_TYPE_MOTION_ESTIMATION_INTEL.

src_image and the ref_image images should represent 8-bit luminance information. image_channel_order and the image_data_type of src_image/ref_image are restricted as follows:

motion_vector_buffer represents an output vector field of pixel block motion vectors stored linearly in row-major order. Each entry of the buffer is a motion vector (packed as two 16-bit integer values) for the corresponding pixel block. The buffer needs to be sized appropriately such that it fits the results of all pixel blocks of the source image. The number of returned motion vectors per source pixel block is determined by the mb_block_type defined at accelerator creation time. Therefore, the total number of the motion vectors is the number of source (16x16) pixel blocks times the number of returned motion vectors per source block (1, 4, or 16).

This kernel optionally takes a buffer of motion vector predictors via the prediction_motion_vector_buffer kernel argument. In many algorithms the motion vectors from the previous frame are used as prediction vectors for the current frame. Prediction vectors can also be used to estimate the motion vectors of the downscaled input image to implement hierarchical motion estimation algorithms. Essentially, using prediction vectors overcomes the hardware limitation on the maximum search radius, as you can offset the neighborhood to be searched, which can be coupled with a multi-pass approach to enable searching within arbitrary areas.

The application can choose not to provide prediction motion vectors by providing NULL as the arg_value argument to clSetKernelArg(), in which case the prediction motion vectors are implied to be (0,0).

A buffer of per-pixel-block distortion values (or “residuals”) can optionally be returned as well, which provides the sum-of-absolute-differences between best-match source and reference frame pixel blocks that produced the corresponding motion vector. The application can choose not to get the residuals by providing NULL as the arg_value argument to clSetKernelArg(), in which case this information is not returned.

Refer to the extension specification document for details.

The clEnqueueNDRangeKernel() for the built-in kernel returns the usual error codes, augmented with a few VME specific error codes, described in the extension specification document. Particularly notice that this built-in kernel requires the local size to be NULL to let the work-group size be determined at runtime, and it requires 2D ND-range. Otherwise the clEnqueueNDRangeKernel() call fails and returns an error as described in the specification.

Full Frame Motion Estimation Code Example

The following code snippet demonstrates how to set up and queue a simple full-frame motion estimation pass for 16x16 pixel blocks (and a single resulting motion vector per block).

    cl_platform_id platform;
    cl_context context;
    cl_device_id device;
    cl_command_queue queue;

// Initialize OpenCL via selecting Intel platform, create context with GPU device and a queue for the device as usual
…

// Get the func pointers to the accelerator routines 
static clCreateAcceleratorINTEL_fn pfn_clCreateAcceleratorINTEL = (clCreateAcceleratorINTEL_fn)
clGetExtensionFunctionAddressForPlatform(platform, "clCreateAcceleratorINTEL");

// Create the program and the built-in kernel for the motion estimation
cl_program program = clCreateProgramWithBuiltInKernels(context,1,device,"block_motion_estimate_intel",NULL);    
cl_kernel kernel = clCreateKernel(program, "block_motion_estimate_intel", NULL);    

// Create the accelerator for the motion estimation 
    cl_motion_estimation_desc_intel desc = { // VME API configuration knobs
// Num of motion vectors per source pixel block, here a single vector per block
       CL_ME_MB_TYPE_16x16_INTEL,                                     
       CL_ME_SUBPIXEL_MODE_INTEGER_INTEL, // Motion vector precision
// Adjust mode for the residuals, we don't compute them in this tutorial anyway: 
       CL_ME_SAD_ADJUST_MODE_NONE_INTEL,  
       CL_ME_SEARCH_PATH_RADIUS_16_12_INTEL // Search window radius
    };
    cl_accelerator_intel accelerator = 
        pfn_clCreateAcceleratorINTEL(context, 
        CL_ACCELERATOR_TYPE_MOTION_ESTIMATION_INTEL, 
        sizeof(cl_motion_estimation_desc_intel), &desc, 0);

    // Input images
    cl_image_format format = { CL_R, CL_UNORM_INT8 }; // luminance plane
    cl_mem srcImage = clCreateImage2D(context, CL_MEM_READ_ONLY, &format, 
        width, height, 0, pSrcBuf, &err);
    cl_mem refImage = clCreateImage2D(context, CL_MEM_READ_ONLY, &format, 
        width, height, 0, pRefBuf, &err);

    // Compute number of output motion vectors 
    const int mbSize = 16; // size of the (input) pixel motion block
    size_t widthInMB  = (width + mbSize - 1) / mbSize;        
    size_t heightInMB = (height + mbSize - 1) / mbSize;
    // Output buffer for MB motion vectors
    cl_mem outMVBuffer = clCreateBuffer(context, CL_MEM_WRITE_ONLY, 
	widthInMB * heightInMB * sizeof(cl_short2), 0, 0, &err);

    // Setup params for the built-in kernel 
    clSetKernelArg(kernel, 0, sizeof(cl_accelerator_intel), &accelerator);
    clSetKernelArg(kernel, 1, sizeof(cl_mem), &srcImage);
    clSetKernelArg(kernel, 2, sizeof(cl_mem), &refImage);
    clSetKernelArg(kernel, 3, sizeof(cl_mem), NULL); // disable predictor motion vectors
    clSetKernelArg(kernel, 4, sizeof(cl_mem), &outMVBuffer);
    clSetKernelArg(kernel, 5, sizeof(cl_mem), NULL); // disable extra motion block info output

    // Run the kernel
    // Notice that it *requires* to let runtime determine the local size, and requires 2D ndrange
    const size_t originROI[2] = { 0, 0 };
    const size_t sizeROI[2] = { width, height};
    clEnqueueNDRangeKernel(queue, kernel, 2, originROI, sizeROI, NULL, 0, 0, 0);

    // Read resulting motion vectors
    clEnqueueReadBuffer(queue, outMVBuffer, CL_TRUE, 0,
widthInMB * heightInMB * sizeof(cl_short2), pMVOut, 0, 0, 0);

    //clReleaseAcceleratorINTEL(accelerator);
    // Release other resources
    …

Example Results

The pictures below show two frames (reference and source) and computed motion vectors overlaid on the second frame. Specifically, the vectors are rendered as the strokes of the appropriate magnitude. So they point to the new (actually best-matched) pixel blocks positions.

Notice the radial pattern of the motion vectors, which is due to the nature of the transformation between frames (zoom in addition to the camera movement).

VME Performance versus Quality Considerations

You should carefully consider performance versus quality trade-offs when using hardware-assisted VME through the cl_intel_motion_estimation extension. Such trade-offs might be simply a function of the input image size, or of the requested density of motion vectors to be computed. The VME implementation computes motion vectors on a 16x16 source pixel block. You can control the number of output motion vectors to be computed by defining the number of output sub-blocks on each of these 16x16 source pixel blocks. As one would expect, requesting more output motion vectors increases VME computation cost.

More precisely, the mb_block_type field of the cl_motion_estimation_desc_intel (refer to the section on the general accelerator API), which is used during VME accelerator creation, defines the number of “sub-blocks” (and hence of motions vectors) within each 16x16 source pixel block:

It is important to understand that sub-blocks are independent. Thus, the smaller the sub-block, the more likely the VME implementation finds a match. Smaller sub-blocks may be appropriate for compression applications that efficiently encode sub-block differences, but less appropriate for feature tracking applications. Using smaller sub-blocks not only increases VME computation expense cost, it also decreases motion vector field smoothness (motion vector directions seem to look more noisy, see the figure below). Feature tracking applications may benefit from a larger sub-block size because they more closely match feature sizes and motion prediction smoothness is often desirable in such applications. Therefore, using larger sub-blocks decreases VME computation expense and improves application performance.

Below are examples of resulting motion vectors fields with the sub-block size varied:

• 4x4 (16 resulting motion vectors per block), the leftmost image

   

• 8x8 (4 resulting motion vectors per block), the middle image

   

• 16x16 (single motion vector per block), the rightmost image

Sub-block sizes are specified with the mb_block_type field of the cl_motion_estimation_desc_intel, a parameter of the VME accelerator routine. Notice the noisiness of the motion vector field in the leftmost image; the noisiness decreases in images to the right:

Conclusion

Computing motions vectors is a key component of many popular video compression and computer vision algorithms. As it is a computationally-intensive task, pure software implementations might present performance or energy efficiency challenges for some applications. In this article, we introduced a Video Motion Estimation (VME) extension for OpenCL* that leverages hardware-assisted motion vectors estimation. We showed how to employ the set of VME extension host-callable functions for the task of computing motion vectors. Specifically, using this VME extension, one can estimate motion in a frame, while trading off the number of resulting motion vectors against computation cost.

About the Author

Intel, the Intel logo, Iris, Ultrabook, and VTune are trademarks of Intel Corporation in the U.S. and/or other countries.

Copyright © 2013 Intel Corporation. All rights reserved.

*Other names and brands may be claimed as the property of others.

Introduction

The Intel® Graphics Performance Analyzers (Intel® GPA) suite is a set of powerful graphics and gaming analysis tools that are designed to work the way game developers do, saving valuable optimization time by quickly providing actionable data to help developers find performance opportunities from the system level down to the individual draw call.

Intel® GPA now supports Intel® Atom™ based phones running the Google* Android* OS. This version of the toolset allows application and driver engineers to optimize their OpenGL* ES 1.0/2.0 workloads on these phones using your choice of development systems: Windows*, OS X*, or Ubuntu* OS. With this capability, as an Android* developer you can:

• get a real-time view of over two dozen critical system metrics covering the CPU, GPU, and OpenGL* ES API
• conduct a number of graphics pipeline experiments to isolate graphics bottlenecks

To download a free copy of Intel GPA, browse to the Intel GPA Home Page, and click the Download button.

Next Steps

For more details on getting started with Intel GPA on the Android* OS, please refer to this article. You can also find more details on using Intel GPA by browsing the product's online help. The Intel GPA home page also contains links to product information, including information about analyzing DirectX* games on the Windows* OS platform, and related products that work with Intel GPA.

If you want to be notified of Intel GPA product updates, use this link.

As always, we welcome your suggestions, so please let us know what we can do to improve your use of these tools by posting your comments on the Intel GPA Support Forum.

*Other names and brands may be claimed as the property of others.

Introduction

The Intel® Graphics Performance Analyzers (Intel® GPA) suite is a set of powerful graphics and gaming analysis tools that are designed to work the way game developers do, saving valuable optimization time by quickly providing actionable data to help developers find performance opportunities from the system level down to the individual draw call. To download a free copy of Intel GPA, browse to the Intel GPA Home Page, and click the Download button.

Whereas Intel GPA supports the analysis of games and graphics applications on both the Windows* OS and the Android* OS for Intel® Atom™ phones, this article discusses the product's support of the Windows* OS platform. To learn more about Intel GPA on the Android* OS, see this article.

Intel GPA fully supports all client versions of both Microsoft* Windows* 7 and Widows* 8/8.1 OS, including both 32-bit and 64-bit versions of this operating system. However, note that Intel GPA does not support applications using the Microsoft* Windows* 8 RT version of this operating system, or the "starter" or "server" versions. Also, Intel GPA supports the following versions of DirectX*: 9/9Ex, 10.0/10.1, and 11.0. Furthermore, at this time the Intel GPA System Analyzer tool is not able to create frame capture or trace capture files for WIndows* 8/8.1 Store Applications.

To use Intel GPA on the Windows* OS platform, you either can run it in

• single-system mode, where the Intel GPA tools run on the same system as your game; install Intel GPA on this system
• "remote" mode, where your game and the tools run on different systems; install the product on both the client and target systems

However, one big tip for developers -- ensure that you have the latest versions of device drivers for your graphics system, since most Intel® Processor Graphics systems require updated drivers for the Windows*8 OS. Download the latest Intel graphics driver for the Windows* OS from the Intel driver download site.

Where to Next...

The Intel GPA home page also contains links to product information, including online help and release notes, and information about the Android* OS version of the product. Here you'll also find information on other Intel products that work together with Intel GPA.

If you want to be notified of Intel GPA product updates, use this link.

As always, we welcome your suggestions, so please let us know what we can do to improve your use of these tools by posting your comments on the Intel GPA Support Forum.

* Other names and brands may be claimed as the property of others.

Android开发过程中为什么要多线程

我们创建的Service、Activity以及Broadcast均是一个主线程处理，这里我们可以理解为UI线程。但是在操作一些耗时操作时，比如I/O读写的大文件读写，数据库操作以及网络下载需要很长时间，为了不阻塞用户界面，出现ANR的响应提示窗口，这个时候我们可以考虑使用Thread线程来解决。

  Android中使用Thread线程会遇到哪些问题

对于从事过J2ME开发的程序员来说Thread比较简单，直接匿名创建重写run方法，调用start方法执行即可。或者从Runnable接口继承，但对于Android平台来说UI控件都没有设计成为线程安全类型，所以需要引入一些同步的机制来使其刷新，这点Google在设计Android时倒是参考了下Win32的消息处理机制。

postInvalidate()方法

对于线程中的刷新一个View为基类的界面，可以使用postInvalidate()方法在线程中来处理，其中还提供了一些重写方法比如postInvalidate(int left,int top,int right,int bottom) 来刷新一个矩形区域，以及延时执行，比如postInvalidateDelayed(long delayMilliseconds)或postInvalidateDelayed(long delayMilliseconds,int left,int top,int right,int bottom) 方法，其中第一个参数为毫秒，如下：

Handler

当然推荐的方法是通过一个Handler来处理这些，可以在一个线程的run方法中调用handler对象的postMessage或sendMessage方法来实现，Android程序内部维护着一个消息队列，会轮训处理这些，如果你是Win32程序员可以很好理解这些消息处理，不过相对于Android来说没有提供PreTranslateMessage这些干涉内部的方法。

消息的处理者，handler负责将需要传递的信息封装成Message，通过调用handler对象的obtainMessage()来实现。将消息传递给Looper，这是通过handler对象的sendMessage()来实现的。继而由Looper将Message放入MessageQueue中。当Looper对象看到MessageQueue中含有Message，就将其广播出去。该handler对象收到该消息后，调用相应的handler对象的handleMessage()方法对其进行处理。

Handler主要接受子线程发送的数据,并用此数据配合主线程更新UI.
      当应用程序启动时，Android首先会开启一个主线程 (也就是UI线程) , 主线程为管理界面中的UI控件，进行事件分发,比如说,你要是点击一个 Button ,Android会分发事件到Button上，来响应你的操作。  如果此时需要一个耗时的操作，例如:联网读取数据， 或者读取本地较大的一个文件的时候，你不能把这些操作放在主线程中，，如果你放在主线程中的话，界面会出现假死现象,如果5秒钟还没有完成的话，，会收到Android系统的一个错误提示  "强制关闭".  这个时候我们需要把这些耗时的操作，放在一个子线程中,因为子线程涉及到UI更新，，Android主线程是线程不安全的，也就是说，更新UI只能在主线程中更新，子线程中操作是危险的.这个时候，Handler就出现了,来解决这个复杂的问题, 由于Handler运行在主线程中(UI线程中),  它与子线程可以通过Message对象来传递数据,这个时候，Handler就承担着接受子线程传过来的(子线程用sedMessage()方法传弟)Message对象，(里面包含数据)  ,把这些消息放入主线程队列中，配合主线程进行更新UI。

Handler一些特点：handler可以分发Message对象和Runnable对象到主线程中,每个Handler实例,都会绑定到创建他的线程中(一般是位于主线程),
      它有两个作用: (1)安排消息或Runnable在某个主线程中某个地方执行

                           (2)安排一个动作在不同的线程中执行
        Handler中分发消息的一些方法
        post(Runnable)
        postAtTime(Runnable,long)
        postDelayed(Runnable long)
        sendEmptyMessage(int)
        sendMessage(Message)
        sendMessageAtTime(Message,long)
        sendMessageDelayed(Message,long)
      以上post类方法允许你排列一个Runnable对象到主线程队列中,sendMessage类方法,允许你安排一个带数据的Message对象到队列中，等待更新.

Handler实例
    // 子类需要继承Hendler类，并重写handleMessage(Message msg) 方法,用于接受线程数据
     // 以下为一个实例，它实现的功能为 :通过线程修改界面Button的内容

public class MyHandlerActivity extends Activity {

    Button button;

    MyHandler myHandler;

    protected void onCreate(Bundle savedInstanceState) {

        super.onCreate(savedInstanceState);

        setContentview(R.layout.handlertest);

        button = (Button) findViewById(R.id.button);

        myHandler = new MyHandler();

        //当创建一个新的Handler实例时,它会绑定到当前线程和消息的队列中,开始分发数据

        // Handler有两个作用, (1) :定时执行Message和Runnalbe对象

        // (2):让一个动作,在不同的线程中执行.

        //它安排消息,用以下方法

        // post(Runnable)

        // postAtTime(Runnable,long)

        // postDelayed(Runnable,long)

        // sendEmptyMessage(int)

        // sendMessage(Message);

        // sendMessageAtTime(Message,long)

        // sendMessageDelayed(Message,long)

        //以上方法以 post开头的允许你处理Runnable对象

        //sendMessage()允许你处理Message对象(Message里可以包含数据,)

        MyThread m = new MyThread();

        new Thread(m).start();

    }

    /**

     *接受消息,处理消息 ,此Handler会与当前主线程一块运行

     * */

    class MyHandler extends Handler {

        public MyHandler() {

        }

        public MyHandler(Looper L) {

            super(L);

        }

        //子类必须重写此方法,接受数据

        @Override

        public void handleMessage(Message msg) {

            // TODO Auto-generated method stub

            Log.d("MyHandler", "handleMessage......");

            super.handleMessage(msg);

            //此处可以更新UI

            Bundle b = msg.getData();

            String color = b.getString("color");

            MyHandlerActivity.this.button.append(color);

        }

    }

    class MyThread implements Runnable {

        public void run() {

            try {

                Thread.sleep(10000);

            } catch (InterruptedException e) {

                // TODO Auto-generated catch block

                e.printStackTrace();

            }

            Log.d("thread.......", "mThread........");

            Message msg = new Message();

            Bundle b = new Bundle();//存放数据

            b.putString("color", "我的");

            msg.setData(b);

            MyHandlerActivity.this.myHandler.sendMessage(msg); //向Handler发送消息,更新UI

        }

    }
}

  Looper

其实Android中每一个Thread都跟着一个Looper，Looper可以帮助Thread维护一个消息队列，昨天的问题 Can't create handler inside thread 错误 一文中提到这一概念，但是Looper和Handler没有什么关系，我们从开源的代码可以看到Android还提供了一个Thread继承类HanderThread可以帮助我们处理，在HandlerThread对象中可以通过getLooper方法获取一个Looper对象控制句柄，我们可以将其这个Looper对象映射到一个Handler中去来实现一个线程同步机制，Looper对象的执行需要初始化Looper.prepare方法就是昨天我们看到的问题，同时推出时还要释放资源，使用Looper.release方法。

Looper是MessageQueue的管理者。每一个MessageQueue都不能脱离Looper而存在，Looper对象的创建是通过prepare函数来实现的。同时每一个Looper对象和一个线程关联。通过调用Looper.myLooper()可以获得当前线程的Looper对象 
创建一个Looper对象时，会同时创建一个MessageQueue对象。除了主线程有默认的Looper，其他线程默认是没有MessageQueue对象的，所以，不能接受Message。如需要接受，自己定义一个Looper对象(通过prepare函数),这样该线程就有了自己的Looper对象和MessageQueue数据结构了。 
Looper从MessageQueue中取出Message然后，交由Handler的handleMessage进行处理。处理完成后，调用Message.recycle()将其放入Message Pool中。

Message

对于Android中Handler可以传递一些内容，通过Bundle对象可以封装String、Integer以及Blob二进制对象，我们通过在线程中使用Handler对象的    sendEmptyMessage或sendMessage方法来传递一个Bundle对象到Handler处理器。对于Handler类提供了重写方法handleMessage(Message msg) 来判断，通过msg.what来区分每条信息。将Bundle解包来实现Handler类更新UI线程中的内容实现控件的刷新操作。相关的Handler对象有关消息发送sendXXXX相关方法如下，同时还有postXXXX相关方法，这些和Win32中的道理基本一致，一个为发送后直接返回，一个为处理后才返回。

Message：消息对象，Message Queue中的存放的对象。一个Message Queue中包含多个Message。 Message实例对象的取得，通常使用Message类里的静态方法obtain(),该方法有多个重载版本可供选择；它的创建并不一定是直接创建一个新的实例，而是先从Message Pool(消息池)中看有没有可用的Message实例，存在则直接取出返回这个实例。如果Message Pool中没有可用的Message实例，则才用给定的参数创建一个Message对象。调用removeMessages()时，将Message从Message Queue中删除，同时放入到Message Pool中。除了上面这种方式，也可以通过Handler对象的obtainMessage()获取一个Message实例。

MessageQueue

是一种数据结构，见名知义，就是一个消息队列，存放消息的地方。每一个线程最多只可以拥有一个MessageQueue数据结构。 
创建一个线程的时候，并不会自动创建其MessageQueue。通常使用一个Looper对象对该线程的MessageQueue进行管理。主线程创建时，会创建一个默认的Looper对象，而Looper对象的创建，将自动创建一个Message Queue。其他非主线程，不会自动创建Looper，要需要的时候，通过调用prepare函数来实现。 
java.util.concurrent对象分析

对于过去从事Java开发的程序员不会对Concurrent对象感到陌生吧，他是JDK 1.5以后新增的重要特性作为掌上设备，我们不提倡使用该类，考虑到Android为我们已经设计好的Task机制，我们这里Android开发网对其不做过多的赘述。

Task以及AsyncTask

在Android中还提供了一种有别于线程的处理方式，就是Task以及AsyncTask，从开源代码中可以看到是针对Concurrent的封装，开发人员可以方便的处理这些异步任务。 当然涉及到同步机制的方法和技巧还有很多，考虑时间和篇幅问题不再做过多的描述。

Downloads

How to Integrate Intel® Perceptual Computing SDK with Cocos2D-x [PDF 482KB]

Introduction

In this article, we will explain the project we worked on as part of the Intel® Perceptual Computing Challenge Brazil, where we managed to achieve 7th place. Our project was Badaboom, a rhythm game set in the Dinosaur Era where the player controls a caveman, named Obo, by hitting bongos at the right time. If you’re curious to see the game in action, check out our video of Badaboom:

​

To begin, you’ll need to understand a bit about Cocos2D-X, an open-source game engine that is widely used to create games for iPhone* and Android*. The good thing about Cocos2D-X is that it is cross-platform and thus is used to create apps for Windows* Phone, Windows 8, Win32*, Linux*, Mac*, and almost any platform you can think of. For more information, go to www.cocos2dx.org.

We will be using the C++ version of the SDK (Version 9302) as well as the Cocos2D-X v2.2 (specifically the Win32 build with Visual Studio* 2012). Following the default pattern of Cocos2D, we will create a wrapper that receives and processes the data from the Creative* Interactive Gesture Camera and interprets it as “touch” for our game.

Setting the environment

To start, you’ll need to create a simple Cocos2D project. We will not cover this subject as it is not the focus of our article. If you need more information, you can find it on the Cocos2D wiki (www.cocos2dx.org/wiki).

To keep it simple, execute the Python* script to create a new project in the “tools” folder of Cocos2d-x and open the Visual Studio project. Now we will add the Intel Perceptual Computing SDK to the project.

To handle the SDK’s input, we will create a singleton class named CameraManager. This class starts the camera, updates the cycle, and adds two images to the screen that represent the position of the hands on the game windows.

CameraManager is a singleton class that is derived from UtilPipeline and imports the “util_pipeline.h” file. Here, we need to reconfigure some of the Visual Studio project properties. Figure 1 shows how to add the additional include directories for the Intel Perceptual Computing SDK.

   $(PCSDK_DIR)/include
   $(PCSDK_DIR)/sample/common/include
   $(PCSDK_DIR)/sample/common/res

Figure 1. Additional include directories

You must also include the following paths to the additional library directories:

   $(PCSDK_DIR)/lib/$(PlatformName)
   $(PCSDK_DIR)/sample/common/lib/$(PlatformName)/$(PlatformToolset)

Figure 2. Additional Library Directories

Add the following dependencies in the input section:

   libpxc_d.lib
   libpxcutils_d.lib

Figure 3. Additional Dependencies

Now we are ready to work on our CameraManager!

Start Coding!

First we need to make the class a singleton. In other words, the class needs to be accessible from anywhere in the code with the same instance (singleton classes have only one instance). For this, you can use a method:

CameraManager* CameraManager::getInstance(void)
{
    if (!s_Instance)
    {
        s_Instance = new CameraManager();
    }

    return s_Instance;
}

After that, we’ll build a constructor, a method that starts the camera:

CameraManager::CameraManager(void)
{
	if (!this->IsImageFrame()){
		this->EnableGesture();

		if (!this->Init()){
			CCLOG("Init Failed");
		}
	}

	this->hand1sprite = NULL;
	this->hand2sprite = NULL;

	hasClickedHand1 = false;
	hasClickedHand2 = false;

	this->inputAreas = CCArray::createWithCapacity(50);
	this->inputAreas->retain();
}

Many of the commands initialize variables that handle sprites which symbolize the users’ hands and get input as they close their hands. The next step is processing the data that comes from the camera.

void CameraManager::processGestures(PXCGesture *gesture){
	
	PXCGesture::Gesture gestures[2]={0};

	gesture->QueryGestureData(0,PXCGesture::GeoNode::LABEL_BODY_HAND_PRIMARY,0,&gestures[0]);
	gesture->QueryGestureData(0,PXCGesture::GeoNode::LABEL_BODY_HAND_SECONDARY,0,&gestures[1]);

	
	CCEGLView* eglView = CCEGLView::sharedOpenGLView();
	switch (gestures[0].label)
	{
	case (PXCGesture::Gesture::LABEL_POSE_THUMB_DOWN):
		CCDirector::sharedDirector()->end();
		break;
	case (PXCGesture::Gesture::LABEL_NAV_SWIPE_LEFT):
		CCDirector::sharedDirector()->popScene();
		break;
	}
}

To be clear, it is in this method that you can also add switch cases to understand voice commands and to implement more gesture handlers. Following this, we must process this information and display it in the CCLayer (Cocos2D sprite layer).

bool CameraManager::Start(CCNode* parent){
	this->parent = parent;

	if (this->hand1sprite!=NULL
    &&  this->hand1sprite->getParent()!=NULL){
		this->hand1sprite->removeFromParentAndCleanup(true);
		this->hand2sprite->removeFromParentAndCleanup(true);
	}

	this->hand1sprite = CCSprite::create("/Images/hand.png");
	this->hand1sprite->setOpacity(150);
    //To make it out of screen
	this->hand1sprite->setPosition(ccp(-1000,-1000));
	this->hand1Pos = ccp(-1000,-1000);
	
	this->hand2sprite = CCSprite::create("/Images/hand.png");
	this->hand2sprite->setFlipX(true);
	this->hand2sprite->setOpacity(150);
	this->hand2sprite->setPosition(ccp(-1000,-1000));
	this->hand2Pos = ccp(-1000,-1000);

	parent->addChild(this->hand1sprite, 1000);
	parent->addChild(this->hand2sprite, 1000);
	
	this->inputAreas->removeAllObjects();
	return true;
}

This method should be called each time a new frame is placed on the screen (most of the time into the onEnter callback). It will automatically remove the hand sprites from the previous parent and add them to the new CCLayer.

Now that our hand sprites have been added to the CCLayer we are able to handle their position by calling the follow method on the update cycle of the CCLayer (which is scheduled by the call: “this->scheduleUpdate();”). The update method is as follows:

void CameraManager::update(float dt){

	if (!this->AcquireFrame(true)) return;

	PXCGesture *gesture=this->QueryGesture();
	
	this->processGestures(gesture);

	PXCGesture::GeoNode nodes[2][1]={0};
	
    gesture->> QueryNodeData(0,PXCGesture::GeoNode::LABEL_BODY_HAND_PRIMARY,1,nodes[0]);
	gesture-> QueryNodeData(0,PXCGesture::GeoNode::LABEL_BODY_HAND_SECONDARY,1,nodes[1]);

	CCSize _screenSize = CCDirector::sharedDirector()->getWinSize();

	
	if (nodes[0][0].openness<20 && !this->hand1Close){
		this->hand1sprite->removeFromParentAndCleanup(true);
		this->hand1sprite = CCSprite::create("/Images/hand_close.png");
		this->hand1sprite->setOpacity(150);
		this->parent->addChild(hand1sprite);
		this->hand1Close = true;
	} else if (nodes[0][0].openness>30 && this->hand1Close) {
		this->hand1sprite->removeFromParentAndCleanup(true);
		this->hand1sprite = CCSprite::create("/Images/hand.png");
		this->hand1sprite->setOpacity(150);
		this->parent->addChild(hand1sprite);
		this->hand1Close = false;
	}
	
	if (nodes[1][0].openness<20 && !this->hand2Close){
		this->hand2sprite->removeFromParentAndCleanup(true);
		this->hand2sprite = CCSprite::create("/Images/hand_close.png");
		this->hand2sprite->setFlipX(true);
		this->hand2sprite->setOpacity(150);
		this->parent->addChild(hand2sprite);
		this->hand2Close = true;
	} else if (nodes[1][0].openness>30 && this->hand2Close) {
		this->hand2sprite->removeFromParentAndCleanup(true);
		this->hand2sprite = CCSprite::create("/Images/hand.png");
		this->hand2sprite->setFlipX(true);
		this->hand2sprite->setOpacity(150);
		this->parent->addChild(hand2sprite);
		this->hand2Close = false;
	}

	this->hand1Pos = ccp(_screenSize.width*1.5-nodes[0][0].positionImage.x*(_screenSize.width*HAND_PRECISION/320) + 100,
						 _screenSize.height*1.5-nodes[0][0].positionImage.y*(_screenSize.height*HAND_PRECISION/240));
	this->hand2Pos = ccp(_screenSize.width*1.5-nodes[1][0].positionImage.x*(_screenSize.width*HAND_PRECISION/320) - 100,
						 _screenSize.height*1.5-nodes[1][0].positionImage.y*(_screenSize.height*HAND_PRECISION/240));

	if (!hand1sprite->getParent() || !hand2sprite->getParent()){
		return;
	}
	this->hand1sprite->setPosition(this->hand1Pos);
	this->hand2sprite->setPosition(this->hand2Pos);

	
    CCObject* it = NULL;
	CCARRAY_FOREACH(this->inputAreas, it)
	{
		InputAreaObject* area = dynamic_cast<InputAreaObject*>(it);
		this->checkActionArea(area->objPos, area->radius, area->sender, area->method);
	}
			
	this->ReleaseFrame();

}

This code not only handles the position of the sprite, it also sets a different sprite (hand_close.png) if the camera detects that the hand is less than 20% open. In addition to this, there is simple logic to create hand precision, which makes the user input more sensitive and easier to get the edges of the screen. We do this because the Perceptual Camera is not that precise on the edges, and the position of the sprites commonly get crazy when we approach the edge.

Now it is indispensable that we add some ways to handle the input (a closed hand is considered a touch). We need to write a method called “checkActionArea” (called in the update method) and register the actionArea.

void CameraManager::checkActionArea(CCPoint objPos, float radius, CCObject* sender, SEL_CallFuncO methodToCall){

	if (sender==NULL)
		sender = this->parent;

	float distanceTargetToHand = ccpDistance(this->hand1Pos, objPos);
	if (distanceTargetToHand<radius){
		if (this->hand1Close&& !hasClickedHand1){

			this->parent->runAction(CCCallFuncO::create(this->parent, methodToCall, sender));
			hasClickedHand1 = true;
		}
	}
	
	if (!this->hand1Close){
		hasClickedHand1 = false;
	} //TODO: repeat for hand2
}

Follow the method registerActionArea() for the registration of areas:

void CameraManager::registerActionArea(CCPoint objPos, float radius, cocos2d::SEL_CallFuncO methodToCall){

	InputAreaObject* newInputArea = new InputAreaObject(objPos, radius, methodToCall);
	this->inputAreas->addObject(newInputArea);
}

Now it is easy to add the Intel Perceptual Computing SDK to your Cocos2D game!!! Just run:

CameraManager::getInstance()->Start(this);

When entering the Layer, register the objects and methods to be called:

CameraManager::getInstance()->registerActionArea(btn_exit->getPosition(), 150, callfuncO_selector(LevelSelectionScene::backClicked));

About us!

Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.
Copyright © 2013 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.

Intel® Graphics Performance Analyzers (Intel® GPA) FAQ

Table of Contents

General Product Information

• What is Intel GPA, and what do I use it for?
• Which platforms does Intel GPA support?
• Is the product really free?
• How does Intel GPA compare with other Intel analysis products?
• What are the key advantages of Intel GPA?
• Have developers been able to use Intel GPA to improve the performance of "real world" games?
• Where do I find out more information about Intel GPA?

Using Intel® GPA

• How do I start using Intel GPA?
• How difficult is it to learn how to use the product?
• What kinds of problems can Intel GPA find?
• How do the Intel GPA System Analyzer and the Intel GPA Frame Analyzer help identify optimization opportunities in my game?
• How does the Intel GPA Platform Analyzer help identify optimization opportunities in my game?
• Will Intel GPA work with any graphics application?
• Do I have to modify the software for my game in order to be able to use Intel GPA?

Technical Requirements

• What are the Intel® GPA system requirements?
• What graphics devices does Intel® GPA support?
• Does Intel® GPA work with netbook computers?
• Is Intel® GPA Frame Analyzer supported on Windows* 32-bit platforms?

Product Support

• How do I get support for Intel® GPA?
• Will Intel® GPA support all future Intel graphics devices?
• What should I expect to see if I attempt to run Intel® GPA on third-party graphics devices?
• What is your plan for supporting OpenGL*?
• How do I submit suggestions or feedback to the development team?

General Product Information

Q: What is Intel GPA, and what do I use it for?
A: Intel GPA is a powerful, agile tool suite enabling game developers to utilize the full performance potential of their gaming platform, including (though not limited to) Intel® Core^TM and Intel® HD Graphics, as well as Intel phones running the Android* OS. Intel GPA visualizes performance data from your application, enabling you to understand system-level and individual frame performance issues, as well as allowing you to perform 'what-if' experiments to estimate potential performance gains from optimizations.

Q: Which platforms does Intel GPA support?
A: Intel GPA supports multiple analysis and target operating systems. Here is a table summarizing the supported platforms:

See the Intel GPA Release Notes for detailed product information on each of these platforms.

Q: Is Intel GPA really free?
A: The product is available at no charge for our valued development community -- to download Intel GPA visit the Intel GPA Home Page.

Q: How does Intel GPA compare with other Intel products such as Intel® VTune^TM Performance Analyzer and Intel® Parallel Studio, and how do I select the right one for my analysis/optimization needs?
A: Intel GPA offers complementary profiling capabilities to other Intel tools focused on debugging and deep hotspot analysis. Intel GPA can help determine whether potential performance bottlenecks exist, and offers the ability to perform "what if" experiments to help optimize the graphics portion of your application. For even deeper performance analysis you can use Intel VTune Amplifier XE with Intel GPA to fine-tune games and media for optimal performance, ensuring cores are fully exploited and new processor capabilities are supported to the fullest.

Q: What are the key advantages of Intel GPA?
A: Intel has worked extensively with game developers to create a product that precisely meets their needs, so they can quickly optimize games. The key advantages of using Intel GPA are:

• Intuitive interface: Quickly find issues, without a lot of clutter; the product's easy workflow fits the way game developers want to optimize their games.
• In-depth, real-time analysis: Identify bottlenecks, experiment with changes, and see results in real time - all within Intel GPA and without modifying the game code.
• Multiple platform support: Optimize games and graphics-intensive applications for Intel systems utilizing processor graphics, or Intel Atom phones running the Android* OS. When possible, Intel GPA accesses hardware metrics in these devices for more accurate measurements of the game's use of the rendering pipeline.
• Task timeline visualization: Use the Intel® GPA Platform Analyzer to see how your task system is balanced (or not) across multiple threads on both the CPU and GPU.

Q: Have developers been able to use Intel GPA to improve the performance of "real world" games?
A: Many developers have utilized Intel GPA to improve game performance on PC's utilizing Intel processor graphics. Many of these games can be found in the Game Gallery. Just a few examples of Intel GPA-enabled titles are Need For Speed:World *, DarkSpore*, LEGO Universe*, Civilization V*, Stalker:Call of Pripyat*, Demigod*, EmpireNapoleon:Total War*, and Ghostbusters, The Video Game*. The performance gains in these games resulted in an increased frame rate and/or additional game features that improved the user experience.

Q: Where do I find out more information about Intel GPA?
A: To find out more about the Intel GPA tool suite, visit the Intel GPA Home Page. The product's home site provides detailed information about the tool, including information on how to download the tool, training and support resources, and videos on the product to help you get started quickly.

Using Intel® GPA

Q: How do I start using Intel GPA?
A: It is pretty easy to get started with Intel GPA -- most game developers start using Intel GPA immediately after installing the package, since Intel GPA uses standard graphics drivers and does not require modifications to your game code (one exception is if you are trying to perform thread-based analysis with the Intel® GPA Platform Analyzer, which requires that you add some code to designate individual threads). To get you up and running quickly, check out the Intel GPA Getting Started Guide, which shows you how to run the main features of the product.

Q: How difficult is it to learn how to use the product?
A: The Intel GPA product features an intuitive user interface that does not require extensive training to quickly access key performance metrics. Therefore, many users will immediately realize many benefits of the product. However, as Intel GPA enables you to perform precise analysis and experiments for every portion of the rendering pipeline, users with a detailed knowledge of Microsoft* DirectX* can quickly utilize even these advanced features.

Q: What kinds of problems can Intel GPA find?
A: If you have performance "hot spots" within your game, Intel GPA can help pinpoint them at the system level, at the frame or sub-frame level, or by visualizing task performance across the CPU/GPU. Once you have identified issues, try different experiments to see the resulting changes in the rendering time as well as the visual effect of these changes. The benefit is that Intel GPA can help improve your frame rate and/or enable you to add new visual effects, while still providing an acceptable level of user interactivity.

Q: How do the Intel GPA System Analyzer and Intel GPA Frame Analyzer help identify optimization opportunities in my game?
A: The Intel GPA System Analyzer application provides access to system-wide metrics for your game, including CPU, GPU, API, and the graphics driver. The metrics available will vary depending upon your platform, but for both Microsoft* Windows* and Google* Android* you will find a large collection of useful metrics to help quantify key aspects of your application's use of system resources. Within the Intel GPA System Analyzer you can also perform various "what-if" experiments to diagnose at a high level where your game's performance bottlenecks are concentrated.

• If the Intel GPA System Analyzer finds that your game is CPU-bound, perform additional fine-tuning of your application using one of the Intel performance optimization products such as Intel® Parallel Studio or Intel® VTune^TM Amplifier XE Performance Profiler.
• If the Intel GPA System Analyzer finds that you game is GPU-bound, use the Intel GPA Frame Analyzer to drill down within a single graphics frame to pinpoint specific rendering problems, such as texture bandwidth, pixel shader performance, level-of-detail issues, or other bottlenecks within the rendering pipeline. For example, using the "simple pixel shader" experiment to determine whether shader complexity is a bottleneck.

Q: How does the Intel GPA Platform Analyzer help identify optimization opportunities in my game?
A: The Intel GPA Platform Analyzer visualizes the execution profile of the tasks in your code on the entire PC platform over time on both the CPU and GPU. This helps you understand task-based issues within your game, enabling you to optimize the compute and rendering tasks across both the CPU and GPU. The Intel GPA Platform Analyzer uses trace data collected during the application run to provide a detailed analysis of how your code executes across all threads, and correlates the CPU workload with that on the GPU.

Q: Though Intel GPA seems to be targeting game developers, will Intel GPA work with other graphics applications?
A: Intel GPA is primarily designed to solve the performance optimization needs of game developers, and for media application developers. However, the features of Intel GPA are broad-based for use with any visual computing application. In other words, our expectation is that anyone developing graphics applications, both "expert" and "novice" alike, should be able to take advantage of the analysis and optimization capabilities of the product.

Q: Do I have to modify the software for my game, or install special drivers, in order to be able to use Intel GPA?
A: The Intel GPA System Analyzer and the Intel GPA Frame Analyzer tools can analyze your game without any code modifications or special libraries. This is possible because Intel GPA accesses the CPU, driver, DirectX*, and GPU metrics directly from the game environment, a big plus for common analysis tasks. For more complex, task-based analysis with the Intel GPA Platform Analyzer, you will benefit by inserting Intel® Instrumentation and Tracing Technology API calls (ITT) that tag the various tasks in your game code -- but you will have to do this once, as the ITT library used by Intel GPA is also used by a number of other Intel performance analysis tools.

Technical Requirements

Q: What are the Intel® GPA system requirements?
A: As the specific requirements depend both upon your target platform and analysis platform, read the Intel GPA release notes for detailed system requirements.

Q: What graphics devices does Intel® GPA support?
A: When the target platform is Windows* OS, Intel® GPA supports Intel® HD graphics (including Intel® HD Graphics 2000/3000 or later). Although Intel GPA may work with other graphics devices, they are unsupported, and some features and metrics may not be available on unsupported platforms. For the Google* Android* OS, Intel GPA only supports Intel phones based upon the Intel® Atom™ processor. Other Android* phones or tablets are not supported, and you should not attempt to run Intel GPA on these non-supported Android devices. When using Intel GPA in the client/target mode, the minimum requirements of the client system used to analyze either Windows* or Android* workloads are: Intel Core Processor with a minimum of 2GB of memory, though at least 4GB of memory and a 64-bit OS are highly recommended.

Q: Does Intel® GPA work with netbook computers and Intel Ultrabooks™?
A: Yes, Intel® GPA supports many popular netbook computers and Ultrabook systems. However, lower end systems may not have sufficient resources to run the Intel GPA tools. So if you encounter issues when running Intel GPA System Analyzer HUD or Intel GPA Frame Analyzer, use the client/target ("networked") version of these tools with a more powerful client system. A good suggestion for a client system would be a computer running a 64-bit OS with greater than 4GB of memory.



Q: Is Intel® GPA Frame Analyzer supported on Windows* 32-bit platforms?
A: No - frame captures taken on 32-bit Windows* OS can be opened remotely from Intel GPA Frame Analyzer installed on a 64-bit Windows* OS platform. For more information, see Intel® GPA: Limited support for Windows* OS 32-bit platforms.

Product Support

Q: How do I get support for Intel GPA?
A: The primary support model for Intel GPA is through the product's Support Forum and Knowledge Base articles. At the Support Forum you can ask questions about the product, share your experiences with other users of the product, and ask for assistance should you encounter issues when using Intel GPA. The Knowledge Base contains various technical notes, "tips & tricks", training material, and pointers to other information that may be of interest to both novice and experienced users alike.

Q: Will Intel GPA support all future Intel graphics devices?
A: Intel intends to continue offering the tools that enable developers to take the best advantage of Intel graphics devices, both now and into the future. Intel will continue to identify, with close cooperation from developers, the best tools to enable optimization and performance of these devices.

Q: When analyzing DirectX* applications, what should I expect to see if I attempt to use Intel® GPA on non-supported graphics devices?
A: Features and performance will vary based upon the hardware capabilities of these other configurations. For example, on non-supported Windows* target devices the Intel GPA System Analyzer cannot provide many detailed GPU metrics, but many of the Intel GPA Frame Analyzer functions work on any graphics device (though again you will  typically see fewer GPU metrics on these devices).

Q: What is your plan for supporting OpenGL* on the Windows* platform?
A: We are actively exploring enhancing Intel GPA to support OpenGL* on the Microsoft* Windows* OS -- if you have specific OpenGL* needs or feature requests, we would love to hear from you at the Intel GPA Support Forum. If you are analyzing Android* applications on Intel phones, the product already supports the Open-GL* API on these platforms.

Q: How do I submit suggestions or feedback to the development team?
A: Use the Intel GPA Support Forum to submit suggestions on new features, and/or to comment on the features currently in the product.


* Other names and brands may be claimed as the property of others.

By Stanislav Pavlov

Downloads

Shadow Mapping Algorithm for Android* [PDF 440KB]

"There is no light without shadows" - Japanese proverb

Because shadows in games make them more realistic and interesting, including well-rendered shadows in your games is important. Currently, most games do not have shadows, but this situation is changing. In this paper we will discuss a common method for realizing shadows, called Shadow Mapping.

Shadow Mapping Theory

Shadow mapping is one of the most conventional techniques for shadow generation in real-time applications. The method is based on the observation that whatever can be seen from the position of the light source is lit; the rest is in the shade. The principle of this method consists in comparing the depth of the current fragment in the reference system associated with the light source to that which is closest to the light source geometry.

The algorithm consists of just two stages:

1. The shadow map generation
2. The rendering stage

The algorithm’s main advantage is that is it easy to understand and implement. Its disadvantages include that it requires more CPU and GPU resources and calculations to make a picture more real. As a result of these additional resources, the shadow map in the depth buffer could become slower.

Algorithm Realization

To create a shadow map, it is necessary to render the scene from the position of the light source. Thus, we obtain the shadow map in the depth buffer, which contains depth values ​​closest to the light source geometry. This approach has the advantage of speed, since the depth buffer generation algorithm is implemented in the hardware.

At the final stage, rendering occurs from the camera position. Each point of the scene is translated into the coordinate system of the light source, and then we calculate the distance from this point to the light source. Calculated distance is compared with the value that is stored in the shadow map. If the distance from the point to the light source is more than the value stored in the shadow map, then this point is in the shadow of any object placed in the path of light.

The code in this article uses the Android SDK (ver. 20) and the Android NDK (ver 8d). It is taken as the basis for a fully native application:  http://developer.android.com/reference/android/app/NativeActivity.html

The Android MegaFon Mint* smartphone is based on the Intel® Atom™ processor Z2460: http://download.intel.com/newsroom/kits/ces/2012/pdfs/AtomprocessorZ2460.pdf

Initialization

The shadow map is stored in a separate texture format GL_DEPTH_COMPONENT, size 512x512 (shadowmapSize.x = shadowmapSize.y = 512), 32 bits per texel (GL_UNSIGNED_INT). In order to optimize, you can use 16 bit textures (GL_UNSIGNED_SHORT). Creating a texture is possible on devices supporting GL_OES_depth_texture [for documentation see http://www.khronos.org/registry/gles/extensions/OES/OES_depth_texture.txt].

The parameters of  GL_TEXTURE_WRAP_S and GL_TEXTURE_WRAP_T are set in GL_CLAMP_TO_EDGE. So when you request any value outside the texture sampling mechanism (sampler), a value corresponding to the boundary is returned. This is done to reduce artifacts from the shadows in the final rendering stage. "Tricks with the fields" will be discussed in another blog.

        //Create the shadow map texture
	glGenTextures(1, &m_textureShadow);
	glBindTexture(GL_TEXTURE_2D, m_textureShadow);
	checkGlError("bind texture");
	// Create the depth texture.
	glTexImage2D(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT, shadowmapSize.x, shadowmapSize.y, 0, GL_DEPTH_COMPONENT, GL_UNSIGNED_INT, NULL);
	checkGlError("image2d");
	// Set the textures parameters
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
	// create frame buffer object for shadow pass
	glGenFramebuffers(1, &m_fboShadow);
	glBindFramebuffer(GL_FRAMEBUFFER, m_fboShadow);
	glFramebufferTexture2D(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_TEXTURE_2D, m_textureShadow, 0);
	checkGlError("shadowmaptexture");
	status = glCheckFramebufferStatus(GL_FRAMEBUFFER);
	if(status != GL_FRAMEBUFFER_COMPLETE) {
		LOGI("init: ");
		LOGI("failed to make complete framebuffer object %xn", status);
	}
	glBindFramebuffer(GL_FRAMEBUFFER, 0);

The next initialization phase is the preparation of shaders.

Below is the Vertex shader attribute. This is the next step of generating shadow maps:

vec3 Position;

uniform mat4 Projection;
uniform mat4 Modelview;

void main(void)
{
	gl_Position = Projection * Modelview * vec4(Position, 1);
}

Pixel shader (step shadow generation):
highp vec4 Color = vec4(0.2, 0.4, 0.5, 1.0);

void main(void)
{
	gl_FragColor = Color;
}

The main task of shaders is to write the geometry, or in other words, to generate the depth buffer for the main stage.

Stages of shadow map rendering

These steps differ from the usual stages of the vectorization scene by the next few points:

1. FBO, which acts as our depth buffer, is attached to the texture (shadow map) glBindFramebuffer (GL_FRAMEBUFFER, m_fboShadow).
2. You can render shadows using orthographic projection from directional sources (the sun), or from a conical (omni) perspective. In the example, the chosen perspective projection matrix lightProjectionMatrix has a wide viewing angle—90 degrees.
3. The color entry in the frame buffer is from the glColorMask (GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE). This optimization can be very useful if you use a complex pixel shader.
4. At this stage, the map is drawn only for the rear surface of the polygons, glCullFace (GL_FRONT). This is one of the most effective and easiest methods to reduce the negative effects of artifacts on the shadow map method. (Note: this is not useful for all geometries.)
5. Area will draw 1 pixel on each side is smaller than the shadow map glViewport ( 0, 0 , shadowmapSize.x - 2 , shadowmapSize.y - 2). This is done in order to leave the "field" on the shadow map.
6. After drawing all the elements of the scene, we return to its original value glCullFace (GL_BACK), glBindFramebuffer (GL_FRAMEBUFFER, 0) and glColorMask (GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE).

void RenderingEngine2::shadowPass() {
	GLenum status;
	glEnable(GL_DEPTH_TEST);
	glBindFramebuffer(GL_FRAMEBUFFER, m_fboShadow);
	status = glCheckFramebufferStatus(GL_FRAMEBUFFER);
	if (status != GL_FRAMEBUFFER_COMPLETE) {
		LOGE("Shadow pass: ");
		LOGE("failed to make complete framebuffer object %xn", status);
	}
	glClear(GL_DEPTH_BUFFER_BIT);
	glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);

	lightProjectionMatrix = VerticalFieldOfView(90.0,
			(shadowmapSize.x + 0.0) / shadowmapSize.y, 0.1, 100.0);
	lightModelviewMatrix = LookAt(vec3(0, 4, 7), vec3(0.0, 0.0, 0.0), vec3(0, -7, 4));
	glCullFace(GL_FRONT);
	glUseProgram(m_simpleProgram);
	glUniformMatrix4fv(uniformProjectionMain, 1, 0,
			lightProjectionMatrix.Pointer());
	glUniformMatrix4fv(uniformModelviewMain, 1, 0,
			lightModelviewMatrix.Pointer());
	glViewport(0, 0, shadowmapSize.x - 2, shadowmapSize.y - 2);

	GLsizei stride = sizeof(Vertex);
	const vector& objects = m_Scene.getModels();
	const GLvoid* bodyOffset = 0;
	for (int i = 0; i < objects.size(); ++i) {
		lightModelviewMatrix = objects[i].m_Transform * LookAt(vec3(0, 4, 7), vec3(0.0, 0.0, 0.0), vec3(0, -7, 4));
		glUniformMatrix4fv(uniformModelviewMain, 1, 0, lightModelviewMatrix.Pointer());
		glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, objects[i].m_indexBuffer);
		glBindBuffer(GL_ARRAY_BUFFER, objects[i].m_vertexBuffer);

		glVertexAttribPointer(attribPositionMain, 3, GL_FLOAT, GL_FALSE, stride,
				(GLvoid*) offsetof(Vertex, Position));

		glEnableVertexAttribArray(attribPositionMain);

		glDrawElements(GL_TRIANGLES, objects[i].m_indexCount, GL_UNSIGNED_SHORT,
				bodyOffset);

		glDisableVertexAttribArray(attribPositionMain);
	}
	glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);

	glBindFramebuffer(GL_FRAMEBUFFER, 0);
	glCullFace(GL_BACK);
}

Rendering scenes with shadows

The first stage of this specific feature is to set textures with the shadow map obtained in the previous step:

glActiveTexture(GL_TEXTURE0);
	glBindTexture(GL_TEXTURE_2D, m_textureShadow);
	glUniform1i(uniformShadowMapTextureShadow, 0);

void RenderingEngine2::mainPass() {
	glClearColor(0.5f, 0.5f, 0.5f, 1);
	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
	modelviewMatrix = scale * rotation * translation
			* LookAt(vec3(0, 8, 7), vec3(0.0, 0.0, 0.0), vec3(0, 7, -8));
	lightModelviewMatrix = LookAt(vec3(0, 4, 7), vec3(0.0, 0.0, 0.0), vec3(0, -7, 4));

	projectionMatrix = VerticalFieldOfView(45.0, (screen.x + 0.0) / screen.y, 0.1, 100.0);
	mat4 offsetLight = mat4::Scale(0.5f) * mat4::Translate(0.5, 0.5, 0.5);
	mat4 lightMatrix = lightModelviewMatrix * lightProjectionMatrix	* offsetLight;
	glUseProgram(m_shadowMapProgram);
	glUniformMatrix4fv(uniformLightMatrixShadow, 1, 0, lightMatrix.Pointer());
	glUniformMatrix4fv(uniformProjectionShadow, 1, 0, projectionMatrix.Pointer());
	glUniformMatrix4fv(uniformModelviewShadow, 1, 0, modelviewMatrix.Pointer());

	glViewport(0, 0, screen.x, screen.y);

	glActiveTexture(GL_TEXTURE0);
	glBindTexture(GL_TEXTURE_2D, m_textureShadow);
	glUniform1i(uniformShadowMapTextureShadow, 0);

	GLsizei stride = sizeof(Vertex);
	const vector& objects = m_Scene.getModels();
	const GLvoid* bodyOffset = 0;
	for (int i = 0; i < objects.size(); ++i) {
		modelviewMatrix = scale * rotation * translation * LookAt(vec3(0, 8, 7), vec3(0.0, 0.0, 0.0), vec3(0, 7, -8));
		glUniformMatrix4fv(uniformTransformShadow, 1, 0, objects[i].m_Transform.Pointer());
		glUniformMatrix4fv(uniformModelviewShadow, 1, 0, modelviewMatrix.Pointer());
		glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, objects[i].m_indexBuffer);
		glBindBuffer(GL_ARRAY_BUFFER, objects[i].m_vertexBuffer);

		glVertexAttribPointer(attribPositionShadow, 3, GL_FLOAT, GL_FALSE,
				stride, (GLvoid*) offsetof(Vertex, Position));
		glVertexAttribPointer(attribColorShadow, 4, GL_FLOAT, GL_FALSE, stride,
				(GLvoid*) offsetof(Vertex, Color));
		glVertexAttribPointer(attribNormalShadow, 3, GL_FLOAT, GL_FALSE, stride,
				(GLvoid*) offsetof(Vertex, Normal));
		glVertexAttribPointer(attribTexCoordShadow, 2, GL_FLOAT, GL_FALSE,
				stride, (GLvoid*) offsetof(Vertex, TexCoord));

		glEnableVertexAttribArray(attribPositionShadow);
		glEnableVertexAttribArray(attribNormalShadow);
		glEnableVertexAttribArray(attribColorShadow);
		glEnableVertexAttribArray(attribTexCoordShadow);

		glDrawElements(GL_TRIANGLES, objects[i].m_indexCount, GL_UNSIGNED_SHORT,
				bodyOffset);

		glDisableVertexAttribArray(attribColorShadow);
		glDisableVertexAttribArray(attribPositionShadow);
		glDisableVertexAttribArray(attribNormalShadow);
		glDisableVertexAttribArray(attribTexCoordShadow);
	}
}

The most interesting parts of these rendering shadows are the shaders. Here’s the technique.

Vertex shader (draws shadows):

attribute vec3 Position;
attribute vec3 Normal;
attribute vec4 SourceColor;
attribute vec2 TexCoord;

varying vec4 fColor;
varying vec3 fNormal;
varying vec2 fTexCoord;
varying vec4 fShadowMapCoord;

uniform mat4 Projection;
uniform mat4 Modelview;
uniform mat4 lightMatrix;
uniform mat4 Transform;

void main(void)
{
	fColor = SourceColor;
	gl_Position = Projection * Modelview * Transform * vec4(Position, 1.0);
	fShadowMapCoord = lightMatrix * Transform * vec4(Position, 1.0);
	fNormal = normalize(Normal);
	fTexCoord = TexCoord;
}

The vertex shader, in parallel with its usual work, produces a translation of vertices in the plane of the light source. In this example, the transition to the plane of the light given by the matrix lightMatrix and the result are passed to the pixel shader through fShadowMapCoord.

Pixel shader (draws shadows):

uniform highp sampler2D shadowMapTex;

varying lowp vec4 fColor;
varying lowp vec3 fNormal;
varying highp vec2 fTexCoord;
varying highp vec4 fShadowMapCoord;

highp vec3 Light = vec3(0.0, 4.0, 7.0);
highp vec4 Color = vec4(0.2, 0.4, 0.5, 1.0);

void main(void)
{
	const lowp float fAmbient = 0.4;
	Light = normalize(Light);
	highp float depth = (fShadowMapCoord.z / fShadowMapCoord.w);
	highp float depth_light = texture2DProj(shadowMapTex, fShadowMapCoord).r;
	highp float visibility = depth <= depth_light ? 1.0 : 0.2;
	gl_FragColor = fColor * max(0.0, dot(fNormal, Light)) * visibility;
}

The pixel shader calculates each pixel value depth based on the relative light source and compares it with the value corresponding to it in the depth map. If the value does not exceed the depth of the depth maps, it is visible from the source position; otherwise, it is in the shade. In this example, we change the visual color intensity using the coefficient visibility, but in general, it is a more difficult technique.

About the Authors

Stanislav works in the Software & Service Group at Intel Corporation. He has 10+ years of experience in software development. His main interest is optimization of performance, power consumption, and parallel programming. In his current role as an Application Engineer providing technical support for Intel® processor-based devices, Stanislav works closely with software developers and SoC architects to help them achieve the best possible performance on Intel platforms. Stanislav holds a Master's degree in Mathematical Economics from the National Research University Higher School of Economics.

Iliya, co-author of this blog, is also a Senior Software Engineer in the Software & Service Group at Intel. He is a developer on the Intel® VTune™ Amplifier team. He received a Master’s degree from the Nizhniy Novgorod State Technical University.

Intel® GPA 2013 R4 now supports tracing of Android* applications. This feature may cause some overhead during the analysis with Intel® GPA System Analyzer, which may impact the accuracy of some platform metrics displayed. If you do not intend to use trace analysis for Android* applications and would like to avoid the possible overhead, you have an option to disable this feature once Intel GPA is installed. 
Note: On Ubuntu* and OS X* analysis systems, tracing of Android* applications is disabled by default.

To disable tracing on a Windows* OS analysis system:

1. Make sure that Intel GPA System Analyzer is not running.
2. Locate the GpaSystemAnalyzer.cfg configuration file in the Intel GPA home directory. On Windows* OS, the default location is: C:\Users\<user_ID>\Documents\<GPA_version>\. 
   The GpaSystemAnalyzer.cfg file appears after the first launch of the Intel GPA System Analyzer. If you do not see this file, run the tool, close it, and try again.
3. Open the configuration file in a text editor.
4. In the Android* section of the file, replace ”install_tracing = true,” with ”install_tracing = false,”. Make sure to preserve the comma at the end of the line. 
   Tracing is disabled until you restore the original setting of the “install_tracing” variable.

Next Steps

For more details on getting started with Intel GPA on the Android* OS, please refer to the product's online help. The Intel GPA home page also contains links to product information, including information about analyzing DirectX* games on the Windows* OS platform.

If you want to be notified of Intel GPA product updates, use this link.

As always, we welcome your suggestions, so please let us know what we can do to improve your use of these tools by posting your comments on the Intel GPA Support Forum.

*Other names and brands may be claimed as the property of others.

Texto de Mitikazu Lisboa, CEO - Hive Digital Media.

A Hive Digital Media é a maior desenvolvedora brasileira de games, e parceira de longa data da Intel no desenvolvimento de diversos projetos, mas mesmo assim ficamos surpresos quando tivémos a demanda de criar um advergame que tivesse talvez a missão mais ambiciosa de qualquer projeto desta natureza que eu já houvesse ouvido falar: Juntar todas as eras dos games em um único advergame que funcionaria como um tributo aos games e gamers de todas as gerações, além de promover as vantagens do Ultrabook.

O processo de desenvolvimento do game foi muito agilizado pela solidez da parceira da Hive com a Intel e pelas facilidades do programa voltado para desenvolvedores da mesma, que permitiu um fluxo ágil de informações, bem como a utilização dos recursos específicos do Ultrabook, uma das primeiras guidelines que definimos em conjunto.

A primeira tarefa foi, obviamente, criar um conceito central para o jogo, junto com um storytelling que fizesse sentido. Depois de alguns dias de conversa entre nós, a Intel e a DM9, agência responsável pela campanha de mídia do ultrabook, chegamos ao conceito do Game Hero, onde um personagem, o Jogador, seria aprisionado por um vilão dos jogos clássicos e teria que derrotá-lo para reconstruir seu ultrabook e de quebra, escapar da sua prisão. Tentamos manter um clima nostálgico e leve, e optamos por trazer muitas referências e "easter eggs" de jogos clássicos, que os jogadores mais "old School" com certeza reconhecerão. Outra decisão estratégica foi misturar diversos estilos de jogos em um só para remeter a tendências das épocas que queremos trabalhar, e para isso dividimos o jogo em 4 "mundos" diferentes, sendo que cada um agrupa características de uma das "eras" dos games, indo desde um típico jogo de plataforma dos consoles de 8 bits e chegando em um modernos FPS, acessível apenas para usuários que estivessem jogado em um ultrabook.

É lógico que essas decisões impactam muito no processo de game design e desenvolvimento, principalmente pelas diferentes jogabilidades que deveriam permear a estrutura do jogo, e isso demandou uma equipe multidisciplinar e alguns profissionais que só se envolveram em algumas partes específicas do jogo, onde suas competências eram relevantes para simular o que queríamos naquela "era dos games" específica. Dos mais de 60 colabores da Hive, aproximadamente 15 foram envolvidos direta ou indiretamente no projeto, fazendo dele um dos nossos maiores cases de 2013.

O principal desafio da construção da área de desenvolvimento foi trabalhar em diferentes plataformas para um mesmo conceito. O diferencial do Game Hero desse ponto de vista é que a base do jogo é executada em uma plataforma e os jogos foram gerados com distribuições em outras tecnologias e fazer essa ponte entre menu , jogos e informações de servidor foi um desafio interessante. Alem disso, a variação de mecânicas e implementação de mashups em um mesmo conceito fez com que a arquitetura do jogo se tornasse complexa, diferenciando-se dos advergames tradicionais.

Para o projeto Game Hero foram utilizadas diversas ferramentas de desenvolvimento de plataformas. O menu principal do jogo, o jogo 8 bits e o jogo 16 bits utilizam o framework Flixel, escrito em ActionScript 3. O jogo 32/bits utiliza a engine Unity3d. O menu do jogo faz a transição para os binários desenvolvidos em fiixel e Unity para o mesmo canvas, enquanto o server-side cuida da transição de informações. Alguns plugins como o Tile Ed, Mesh Exploder e Action RPG foram utilizados no jogo 32/64 para auxiliar em alguns mecanismos necessários. Para o mundo moderno, estamos utilizando Unity3d também, com a diferença de ser um standalone que será baixado pelo usuário e instalado no computador pessoal. O resultado final ficou bem sólido, e também oferece um desafio digno de qualquer old school gamer que queira se arriscar em uma aventura pelas diversas eras do game.

Veja abaixo o trailer do Game Hero e entenda porque esse é um projeto que passa por todas as Eras dos Games.

Se interessou? Gostaria de saber mais? Então acesse os links abaixo:

1. Website do GAME HERO
2. Website da Hive Digital Media
3. Desenvolvimento de aplicativos para dispositivos Intel
4. Como se filiar ao programa de software da Intel
5. Ferramentas para jogos e mídia

Texto de Mitikazu Lisboa (CEO - Hive Digital Media).

A Hive Digital Media é a maior desenvolvedora brasileira de games, e parceira de longa data da Intel no desenvolvimento de diversos projetos, mas mesmo assim ficamos surpresos quando tivémos a demanda de criar um advergame que tivesse talvez a missão mais ambiciosa de qualquer projeto desta natureza que eu já houvesse ouvido falar: Juntar todas as eras dos games em um único advergame que funcionaria como um tributo aos games e gamers de todas as gerações, além de promover as vantagens do Ultrabook.

O processo de desenvolvimento do game foi muito agilizado pela solidez da parceira da Hive com a Intel e pelas facilidades do programa voltado para desenvolvedores da mesma, que permitiu um fluxo ágil de informações, bem como a utilização dos recursos específicos do Ultrabook, uma das primeiras guidelines que definimos em conjunto.

A primeira tarefa foi, obviamente, criar um conceito central para o jogo, junto com um storytelling que fizesse sentido. Depois de alguns dias de conversa entre nós, a Intel e a DM9, agência responsável pela campanha de mídia do ultrabook, chegamos ao conceito do Game Hero, onde um personagem, o Jogador, seria aprisionado por um vilão dos jogos clássicos e teria que derrotá-lo para reconstruir seu ultrabook e de quebra, escapar da sua prisão. Tentamos manter um clima nostálgico e leve, e optamos por trazer muitas referências e "easter eggs" de jogos clássicos, que os jogadores mais "old School" com certeza reconhecerão. Outra decisão estratégica foi misturar diversos estilos de jogos em um só para remeter a tendências das épocas que queremos trabalhar, e para isso dividimos o jogo em 4 "mundos" diferentes, sendo que cada um agrupa características de uma das "eras" dos games, indo desde um típico jogo de plataforma dos consoles de 8 bits e chegando em um modernos FPS, acessível apenas para usuários que estivessem jogado em um ultrabook.

É lógico que essas decisões impactam muito no processo de game design e desenvolvimento, principalmente pelas diferentes jogabilidades que deveriam permear a estrutura do jogo, e isso demandou uma equipe multidisciplinar e alguns profissionais que só se envolveram em algumas partes específicas do jogo, onde suas competências eram relevantes para simular o que queríamos naquela "era dos games" específica. Dos mais de 60 colabores da Hive, aproximadamente 15 foram envolvidos direta ou indiretamente no projeto, fazendo dele um dos nossos maiores cases de 2013.

O principal desafio da construção da área de desenvolvimento foi trabalhar em diferentes plataformas para um mesmo conceito. O diferencial do Game Hero desse ponto de vista é que a base do jogo é executada em uma plataforma e os jogos foram gerados com distribuições em outras tecnologias e fazer essa ponte entre menu , jogos e informações de servidor foi um desafio interessante. Alem disso, a variação de mecânicas e implementação de mashups em um mesmo conceito fez com que a arquitetura do jogo se tornasse complexa, diferenciando-se dos advergames tradicionais.

Para o projeto Game Hero foram utilizadas diversas ferramentas de desenvolvimento de plataformas. O menu principal do jogo, o jogo 8 bits e o jogo 16 bits utilizam o framework Flixel, escrito em ActionScript 3. O jogo 32/bits utiliza a engine Unity3d. O menu do jogo faz a transição para os binários desenvolvidos em fiixel e Unity para o mesmo canvas, enquanto o server-side cuida da transição de informações. Alguns plugins como o Tile Ed, Mesh Exploder e Action RPG foram utilizados no jogo 32/64 para auxiliar em alguns mecanismos necessários. Para o mundo moderno, estamos utilizando Unity3d também, com a diferença de ser um standalone que será baixado pelo usuário e instalado no computador pessoal. O resultado final ficou bem sólido, e também oferece um desafio digno de qualquer old school gamer que queira se arriscar em uma aventura pelas diversas eras do game.

Veja abaixo o trailer do Game Hero e entenda porque esse é um projeto que passa por todas as Eras dos Games.

Se interessou? Gostaria de saber mais? Então acesse os links abaixo:

1. Website do GAME HERO
2. Website da Hive Digital Media
3. Desenvolvimento de aplicativos para dispositivos Intel
4. Como se filiar ao programa de software da Intel
5. Ferramentas para jogos e mídia

Download Article

Download Gameplay: Touch controls for your favorite games [PDF 703KB]

GestureWorks Gameplay is a revolutionary new way of interacting with popular PC games. Gameplay software for Windows 8 lets gamers use and build their own Virtual Controllers for touch, which are overlaid on top of existing PC games. Each Virtual Controller overlay adds buttons, gestures, and other controls that are mapped to input the game already understands. In addition, gamers can use hundreds of personalized gestures to interact on the screen. Ideum’s collaboration with Intel gave them access to technology and engineering resources to make the touch overlay in Gameplay possible.

Check out this one-minute video that explains the Gameplay concept.

It’s all about the virtual controllers

Unlike traditional game controllers, virtual controllers can be fully customized and gamers can even share them with their friends. Gameplay works on Windows 8 tablets, Ultrabooks, 2-in-one laptops, All-In-Ones, and even multitouch tables and large touch screens.

Figure 1 - Gameplay in action on Intel Atom-based tablet

"The Virtual Controller is real! Gameplay extends hundreds of PC games that are not touch-enabled and it makes it possible to play them on a whole new generation of portable devices, " says Jim Spadaccini, CEO of Ideum, makers of GestureWorks Gameplay. "Better than a physical controller, Gameplay’s Virtual Controllers are customizable and editable. We can’t wait to see what gamers make with Gameplay."

Figure 2 - The Home Screen in Gameplay

Several dozen pre-built virtual controllers for popular Windows games come with GestureWorks Gameplay (currently there are over 116 unique titles). Gameplay lets users configure, layout, and customize existing controllers as well. The software also includes an easy to use, drag-and-drop authoring tool allowing users to build their own virtual controller for many popular Windows-based games distributed on the Steam service.

Figure 3 - Virtual Controller layout view

Users can place joysticks, D-pads, switches, scroll wheels, and buttons anywhere on the screen, change the size, opacity, and add colors and labels. Users can also create multiple layout views which can be switched in game at any time. This allows a user to create unique views for different activities in game, such as combat versus inventory management functions in a Role Playing Game.

Figure 4 - Virtual Controller Global Gestures View

Powered by the GestureWorks gesture-processing engine aka GestureWorks Core, Gameplay provides support for over 200 global gestures. Basic global gestures such as tap, drag, pinch/zoom, and rotate are supported by default, but are also customizable. This allows extension of overlaid touch controllers, giving gamers access to multi-touch gestures that can provide additional controls to PC games. For example, certain combat moves can be activated with a simple gesture versus a button press in a FPS. Gameplay even includes experimental support for accelerometers so you can steer in a racing game by tilting your Ultrabook™ or tablet, and it detects when you change your 2-1 device to tablet mode to optionally turn on the virtual controller overlay.

Challenges Addressed During Development

Developing all this coolness was not easy, to make the vision for Gameplay a reality, several technical challenges had to be overcome. Some of these were solved using traditional programming methods, while others required more innovative solutions.

DLL injection

DLL injection is a method used for executing code within the address space of another process by getting it to load an external dynamically-linked library. While DLL injection is often used by external programs for nefarious reasons, there are many legitimate uses for it, including extending the behavior of a program in a way its authors did not anticipate or originally intend. With Gameplay, we needed a method to insert data into the input thread of the process (game) being played so the touch input could be translated to inputs the game understood. Of the myriad methods for implementing DLL injection, Ideum chose to use the Windows hooking calls in the SetWindowsHookEx API. Ultimately, Ideum opted to use process-specific hooking versus global hooking for performance reasons.

Launching games from a third-party launcher

Two methods of hooking into a target processes address space were explored. The application can hook into a running process’ address space, or the application can launch the target executable as a child process. Both methods are sound; however, in practice, it is much easier to monitor and intercept processes or threads created by the target process when the application is a parent of the target process.

This poses a problem for application clients, such as Steam and UPlay, that are launched when a user logs in. Windows provides no guaranteed ordering for startup processes, and the Gameplay process must launch before these processes to properly hook in the overlay controls. Gameplay solves this issue by installing a lightweight system service during installation that monitors for startup applications when a user logs in. When one of the client applications of interest starts, Gameplay is then able to hook in as a parent to the process insuring the overlay controls are displayed as intended.

Lessons Learned

Mouse filtering

During development, several game titles were discovered that incorrectly processed virtual mouse input received from the touch screen. This problem largely manifested with First Person Shooter titles or Role Playing Titles that have a "mouse-look" feature. The issue was that the mouse input received from the touch panel was absolute with respect to a point on the display, and thus in the game environment. This made the touch screen almost useless as a "mouse-look" device. The eventual fix was to filter out the mouse inputs by intercepting the input thread for the game. This allowed Gameplay to emulate mouse input via an on-screen control such as a joystick for the "mouse-look" function. It took a while to tune the joystick responsiveness and dead zone to feel like a mouse, but once that was done, everything worked beautifully. You can see this fix in action on games like Fallout: New Vegas or The Elder Scrolls: Skyrim.

Vetting titles for touch gaming

Ideum spent significant amounts of time tuning the virtual controllers for optimal gameplay. There are several elements of a game that determine its suitability for using with Gameplay. Below are some general guidelines that were developed for what types of games work well with Gameplay:

Gameplay playability by game type

While playability is certainly an important aspect of vetting a title for use with Gameplay, the most important criteria is stability. Some titles will just not work with either the hooking technique, input injection, or overlay technology. This can happen for a variety of reasons, but most commonly is due to the game title itself monitoring its own memory space or input thread to check for tampering. While Gameplay itself is a completely legitimate application, it employs techniques that can also be used for the forces of evil, so unfortunately some titles that are sensitive to these techniques will never work unless enabled for touch natively.

User Response

While still early in its release, Gameplay 1.0 has developed some interesting user feedback in regards to touch gaming on a PC. There are already some clear trends to the user feedback being received. At a high-level, it is clear that everyone universally loves being able to customize the touch interface for games. The remaining feedback focuses on personalizing the gaming experience in a few key areas:

• Many virtual controllers are not ideal for left handed people, this was an early change to many of the published virtual controllers.
• Button size and position is the most common change, so much so, that Ideum is considering adding an automatic hand sizing calibration in a future Gameplay release.
• Many users prefer rolling touch inputs vs. discrete touch and release interaction.

We expect many more insights to reveal themselves as the number of user created virtual controllers increases.

Conclusion

GestureWorks Gameplay brings touch controls to your favorite games. It does this via a combination of a visual overlay and supports additional interactions like gesture, accelerometers, and 2-1 transitions. What has been most interesting in working on this project has been the user response. People are genuinely excited about touch-gaming on PCs, and ecstatic they can now play many of the titles they previously enjoyed with touch.

About Erik

Erik Niemeyer is a Software Engineer in the Software & Solutions Group at Intel Corporation. Erik has been working on performance optimization of applications running on Intel microprocessors for nearly fifteen years. Erik specializes in new UI development and micro-architectural tuning. When Erik is not working he can probably be found on top of a mountain somewhere. Erik can be reached at erik.a.niemeyer@intel.com.

About Chris

Chris Kirkpatrick is a software applications engineer working in the Intel Software and Services Group supporting Intel graphics solutions on mobile platforms in the Visual & Interactive Computing Engineering team. He holds a B.Sc. in Computer Science from Oregon State University. Chris can be reached at chris.kirkpatrick@intel.com.

Resources

https://gameplay.gestureworks.com/

http://software.intel.com/en-us/articles/detecting-slateclamshell-mode-screen-orientation-in-convertible-pc

Intel, the Intel logo, and Ultrabook are trademarks of Intel Corporation in the U.S. and/or other countries.

Copyright © 2014 Intel Corporation. All rights reserved.

*Other names and brands may be claimed as the property of others.

Neste tutorial iremos demonstrar como criar um jogo simples usando o framework Cocos2d-x em um ambiente de desenvolvimento Windows* e como compilar o jogo em Windows e Android.

O que é o Cocos2d-x?

O Cocos2d-x é um framework multi-plataforma de desenvolvimento de jogos (e outros apps gráficos como livros interativos) baseado no cocos2d para iOS, mas que usa C++, Javascript ou Lua ao invés do Objective-C.

Uma das vantagens desse framework é permitir o desenvolvimento de jogos para diversas plataformas (Android*, iOS*, Win32*, Windows* Phone, Windows* 8, Mac*, Linux*, etc) mantendo apenas uma base de código (com algumas adaptações sendo feitas de uma plataforma para outra).

Ele é open-source sob a licença MIT License e seu código fonte pode ser encontrado aqui.

Para mais informações sobre o Cocos2d-x e sua documentação, acesse: http://www.cocos2d-x.org/

Criando seu primeiro game

1-  Faça o download da versão mais recente do framework no site e descompacte no seu ambiente de desenvolvimento. Para este tutorial foi utilizada a versão 2.2.2 e o framework foi descompactado na área de trabalho (C:\Users\felipe.pedroso\Desktop\cocos2d-x-2.2)

2- Para criar um novo projeto no cocos2d-x, iremos usar um script python (create_project.py) que cria toda a estrutura do projeto dentro da pasta aonde o framework foi descompactado. Caso não tenha o runtime do Python instalado, faça o download da versão 2.7.6 no seguinte link: http://www.python.org/download/.

3-  Abra o prompt de comando (cmd.exe) e execute os seguintes comandos:

-   Navegue até a pasta do script (é importante que o script create_project.py seja executado de dentro da pasta project-creator)
    
    cd C:\Users\felipe.pedroso\Desktop\cocos2d-x-2.2\tools\project-creator
    
-   Execute o script com seguinte comando:
    
    python create_project.py -project MyFirstGame -package com.example.myfirstgame -language cpp
    

​

Explicando os parâmetros:
    project: Nome do seu projeto/game
    package: Nome do pacote da sua aplicação (ex.: br.suaempresa.MyFirstGame)
    language: Linguagem de programação a ser utilizada (cpp, lua e javascript)

Obs.: para executar o comando 'python' na linha de comando, adicione a pasta aonde o Python foi instalado na variável de ambiente Path. 

-   Se tudo funcionar corretamente, seu projeto será criado na pasta projects, dentro do diretório onde o framework foi descompactado.

O projeto criado contém o código base do jogo (Classes), os recursos (imagens, áudio, etc) e projetos para cada uma das plataformas suportadas pelo framework.

Compilando para Win32 (Windows* 7 ou Windows* 8 modo desktop)

Requisitos:

• Visual Studio 2012 

Dentro do diretório do projeto, abra o arquivo MyFirstGame.sln que está na pasta proj.win32 com o Visual Studio.

Compile o projeto pressionando F6 (ou menu Build -> Build Solution) e execute o projeto pressionando F5 (ou menu Debug->Start Debugging).

Se tudo der certo, você verá a seguinte janela:

Compilando para Windows* 8 (Windows* Store App)

Requisitos:

• Visual Studio 2012 

Para compilar o projeto como uma Windows* Store App, abra o arquivo MyFirstGame.sln da pasta proj.winrt e compile da mesma maneira do projeto Win32.

Após compilar e rodar, você verá a seguinte tela:

Obs.: a versão do cocos2d-x utilizada no tutorial não funcionou no Windows* 8.1

Compilando para Android*

Requisitos

• Android* SDK;
• Android* NDK
• Apache Ant

Da mesma forma que o Python foi adicionado no Path do Windows*, adicione os diretórios tools e platform-tools do Android SDK,o diretório raiz do NDK e o diretório bin do Apache Ant para poder usar os comandos para fazer build do app.

Abra um novo prompt de comando (cmd.exe) e execute os seguintes comandos para configurar as variáveis de ambiente necessárias para a compilação do app Android:

    set COCOS2DX_ROOT=C:\Users\felipe.pedroso\Desktop\cocos2d-x-2.2
    set NDK_TOOLCHAIN_VERSION=4.8
    set NDK_MODULE_PATH=%COCOS2DX_ROOT%;%COCOS2DX_ROOT%\cocos2dx\platform\third_party\android\prebuilt
    
Explicando as variáveis:

    COCOS2DX_ROOT: diretório aonde o cocos2d-x foi extraído
    NDK_TOOLCHAIN_VERSION: versão do toolchain do NDK que será utilizado para compilar o projeto
    NDK_MODULE_PATH: Módulos que devem ser incluídos na compilação do NDK. No caso do exemplo, estamos utilizando os módulos pré-compilados do cocos2d-x

Com as variáveis de ambiente configuradas, navegue até o diretório do projeto:

    cd C:\Users\felipe.pedroso\Desktop\cocos2d-x-2.2\projects\MyFirstGame\proj.android

Copie os resources do jogo (imagens, sons, etc) para a pasta assets:

    rmdir /S /Q assets
    mkdir assets
    xcopy /E ..\Resources .\assets

Execute o seguinte comando para compilar os módulos nativos:

    ndk-build.cmd -C . APP_ABI="armeabi armeabi-v7a x86"

O comando irá gerar as bibliotecas nativas para três arquiteturas diferentes: ARM, ARM-NEON e x86. Isso permitirá que seu jogo rode em diversas arquiteturas aproveitando o que elas oferecem de melhor. .Após finalizar a compilação dos módulos nativos, compile o app Android com o comando ant:

    ant debug

    
Agora para instalar o aplicativo em um device ou emulador use o comando:

    adb install -r bin\MyFirstGame.apk

Depois é só abrir o app:

Pronto, agora seu jogo já pode rodar em pelo menos três plataformas: Android* (em três arquiteturas), Windows* 7 e Windows* 8! 

Até a próxima!

* Other names and brands may be claimed as the property of other

By John Wesolowski

Downloads

How To Plan Optimizations with Unity* [PDF 2.15MB]

Abstract

Unity provides a number of tools and settings to help make games perform smoothly. For this project, we chose ones we thought could prove to be troublesome and analyzed how they affected game performance on Intel® graphics processors.

We put ourselves in the shoes of a game developer learning how to use Unity. We wanted to stumble into performance pitfalls and then determine how to work through issues with Unity’s built-in performance mechanisms. One of Unity’s strengths is the ability to create content quickly, but when considering performance, especially on mobile and tablet devices, the developer needs to slow down and plan out how to utilize the built in performance mechanisms. This paper prepares new and existing Unity users with performance considerations when building your levels/games, and offers new ways to build.

Introduction

Creating games within Unity is fairly simple. Unity offers a store where you can purchase items like meshes, pre-written scripts, game demos, or even full games. For the purposes of my testing, I was concerned with manipulating an existing game to find areas where performance gains could or could not be achieved. I dove into the Unity Tech Demo called Boot Camp, free for download in the assets store, to see what kind of trouble I could get into.

I used Unity 3.0 to create the game settings and run all of the scenes. The testing was performed on a 3rd generation Intel® Core™ processor-based computer with Intel® HD Graphics 4000. The test results are not applicable to mobile devices.

Quality Manager

Unity has extra render settings for games found in: Edit->Project Settings->Quality menu (Figure 1). They are customizable render settings that can be modified for individual needs. Unity has helpful online documentation for explaining what the Quality Settings are and how to modify these settings through Unity’s scripting API.

Figure 1. The Tags and Layers available through the Edit->Project Settings->Tag inspector

As for my task to find optimizations with Unity, I decided to mess around with some of the Quality Settings to see what kind of gains or losses I could find, although I did not test all of the different options available.

Texture Quality

The Quality Settings Inspector has a drop down menu where you select render resolutions for your textures. You can choose from 1/8, ¼, ½, or Full Resolution. To see the performance gains/losses between different texture resolutions, I took frame rate captures of a sample scene, testing all of Unity’s default Quality Settings (Fastest, Fast, Good, etc.), while adjusting only the Texture Quality between each capture. Figures 2 and 3 show a comparison between a scene with 1/8 Texture Resolution and Full Resolution.

Figure 2. Unity* Scene Boot Camp running at 1/8 resolution

Figure 3. Unity* Scene Boot Camp running at full resolution

We took a frames per second (FPS) capture using Intel® Graphics Performance Analyzers (Intel® GPA) after changing the texture resolution. Looking at the Fantastic setting (Table 1), you can see the performance did not change much by varying the texture sizes.

Table 1. Illustrates the change in FPS while switching between Unity’s* provided texture qualities

Although an Intel® graphics-based PC’s performance is not affected by texture size changes, there are other things to consider, like the total amount of memory on the device and its usage by the application.

Shadow Distance

Shadow distance is a setting that changes the culling distance of the camera being used for the shadows of game objects. Game objects within the shadow distance’s value from the camera have their shadows sent for rendering, whereas objects that are not within the shadow distance value do not have their shadows drawn.

Depending on the settings used, shadows can adversely affect performance due the amount of processing they require. To test the impact of Shadow Distance:

• Set up a sample scene
• Set scene to a Unity default quality setting
• Adjust the shadow distance incrementally and take FPS captures using Intel GPA
• Select different Unity default quality settings and repeat shadow distance captures

This test did not use the Fastest and Fast Quality Levels because those default to turning shadows off.

Figure 4. This is a setting found under the Inspector menu of Edit->Project Settings->Quality

Figure 5. Unity* Tech Demo Boot Camp

Table 2. FPS results from changing the Shadow Distance of Unity* Tech Demo, Boot Camp

Shadows significantly impact performance. The data shows the FPS dropped by almost half when going from a distance of 0 to 50 Simple mode. It is important to consider if game objects can actually be seen and to make sure you are not drawing shadows unnecessarily. The shadow distance and other shadow settings can be controlled during gameplay via Unity scripting and can accommodate numerous situations. Although we only tested the effects of shadow distance, we expect similar performance deltas occur when changing the other settings under Shadow in the Quality settings.

Layers

All game objects inside Unity are assigned to a layer upon creation. They are initially assigned to the default layer, as show in Figure 6, but you can create your own unique layers. There are two ways to do this. You can simply click on the box next to Layer and select Add New Layer. You can also go to Edit->Project Settings->Tags.

Figure 6. The Layer menu found inside the Inspector of a game object

Figure 7. The Tag Manager via the inspector menu

From the inspector window (Figure 7) you can create a new layer and specify which layer number you want it to belong to. Both methods lead you to the same Tag Manager window. Once a layer is created, game objects can be assigned to them by choosing the desired layer under the options box next to Layer under that game object’s inspector window. This way, you can group objects in common layers for later use and manipulation. Keep in mind what layers are and how to create and modify them for when I talk about a few other layer features later in the paper.

Layer Cull Distances

Your camera will not render game objects beyond the camera’s clipping plain in Unity. There is a way, through Unity scripting, to have certain layers set to a shorter clipping plane.

Figure 8. Sample script taken from Unity*’s Online Documentation showing how to modify a layer’s culling distance

It takes a bit of work to set up game objects so they have a shorter culling distance. First, place the objects onto a layer. Then, write a script to modify an individual layer’s culling distance and attach it to the camera. The sample script in Figure 8 shows how a float array of 32 is created to correspond to the 32 possible layers available for creation under the Edit->Project Settings->Tags. Modifying a value for an index in your array and assigning it to camera.layerCullDistances will change the culling distance for the corresponding layer. If you do not assign a number for an index, the corresponding layer will use the camera’s far clip plane.

To test performance gains from layerCullDistances, I set up three scenes filled with small, medium, and large objects in terms of complexity. The scenes were arranged with a number of identical game objects grouped together and placed incrementally further and further away from the camera. I used Intel GPA to take FPS captures while incrementing the layer culling distance each time, adding another group of objects to the capture, i.e., the first capture had one group of objects, whereas the sixth capture had six groups of objects.

Figures 9, 10, and 11 show the scenes I used for testing with the different types of objects.

Boots: Poly – 278 Vertices – 218

Figure 9. Test scene filled with low polygon and vertices count boot objects

T-Rex’s: Poly – 4398 Vertices – 4400

Figure 10. Test scene filled with medium polygon and vertices count dinosaur objects

Airplane: Poly - 112,074 Vertices - 65,946

Figure 11. Test scene filled with large polygon and vertices count airplane objects

Tables 3, 4, and 5 show the change in FPS for each of the scenes tested.

Table 3. Data collected from the scene with boots (Figure 9)

Table 4. Data collected from the scene with dinosaurs (Figure 10)

Table 5. Data collected from the scene with airplanes (Figure 11)

Table 6. Fantastic mode data from all the test scenes

This data shows that performance gains can be achieved from using the layerCullDistances feature within Unity.

Table 6 illustrates how having more objects on the screen impacts performance, especially with complex objects. As a game developer, using the layerCullDistances proves to be very beneficial for performance if utilized properly. For example, smaller objects with a complex mesh that are farther away from the camera can be set up to only draw when the camera is close enough for the objects to be distinguished. While planning and designing a level, the developer needs to consider things like mesh complexity and the visibility of objects at a greater distance from the camera. By planning ahead, you can achieve greater benefits from using layerCullDistances.

Camera

I explored Unity’s camera, focusing on its settings and features. I toyed with some of the options under its GUI and examined other features and addons.

Figure 12. The Inspector menu that appears while having a camera selected

When creating a new scene, by default, there is only one camera game object labeled Main Camera. To create or add another camera, first create an empty game object by going to: Game Object->Create Empty. Then select the newly created empty object and add the camera component: Components->Rendering->Camera.

Unity’s camera comes with a host of functionality inside its GUI, as shown in Figure 12. The features I chose to explore were: Rendering Path and HDR.

Render Path

The Render Path tells Unity how to handle light and shadow rendering in the game. Unity offers three render types, listed from highest cost to least; Deferred (Pro Only), Forward, and Vertex Lit rendering. Each renderer handles light and shadow a little bit differently, and they require different amounts of CPU and GPU processing power. It’s important to understand the platform and hardware you want to develop for so you can choose a renderer and build your scene or game accordingly. If you pick a renderer that is not supported by the graphics hardware, Unity will automatically lower the rendering path to a lower fidelity.

Figure 13. Player Settings Inspector window

The Rendering Path can be set in two different ways. The first is under the Edit->Project Settings->Player (Figure 13). You will find the Rendering Path drop down box under the Others Settings tab. The second is from the Camera Inspector GUI (Figure 14). Choosing something other then ‘Use Player Settings’ will override the rendering path set in your player settings, but only for that camera. So it is possible to have multiple cameras using different rendering buffers to draw the lights and shadows.

Figure 14. The drop down box from selecting the Rendering Path under the Camera GUI

Developers should know that these different light rendering paths are included in Unity and how each handles rendering. The reference section at the end of this document has links to Unity’s online documentatoin. Make sure you know your target audience and what type of platform they expect their game to be played on. This knowledge will help you select a rendering path appropriate to the platform. For example, a game designed with numerous light sources and image effects that uses deferred rendering could prove to be unplayable on a computer with a lower end graphics card. If the target audience is a casual gamer, who may not possess a graphics card with superior processing power, this could also be a problem. It is up to developers to know the target platform on which they expect their game to be played and to choose the lights and rendering path accordingly.

HDR (High Dynamic Range)

In normal rendering, each pixel’s red, blue, and green values are represented by a decimal number between 0 and 1. By limiting your range of values for the R, G, and B colors, lighting will not look realistic. To achieve a more naturalistic lighting effect, Unity has an option called HDR, which when activated, allows the number values representing the R, G, and B of a pixel to exceed their normal range. HDR creates an image buffer that supports values outside the range of 0 to 1, and performs post-processing image effects, like bloom and flares. After completing the post-processing effects, the R, G, and B values in the newly created image buffer are reset to values within the range of 0 to 1 by the Unity Image Effect Tonemapping. If Tonemapping is not executed when HDR is included, the pixels could be out of the normal accepted range and cause some of the colors in your scene to look wrong in comparison to others.

Pay attention to a few performance issues when using HDR. If using Forward rendering for a scene, HDR only will be active if image effects are present. Otherwise, turning HDR on will have no effect. Using Deferred rendering supports HDR regardless.

If a scene is using Deferred rendering and has Image Effects attached to a camera, HDR should be activated. Figure 15 compares the draw calls for a scene with image effects and deferred rendering while HDR is turned on and HDR is off. With HDR off and image effects included, you see a larger number of draw calls then if you include image effects with HDR turned on. In Figure 15, the number of draw calls are represented by the individual blue bars, and the height of each blue bar reveals the amount of GPU time each draw call took.

Figure 15. The capture from Intel® Graphics Performance Analyzers with HDR OFF shows over 2000 draw call, whereas the capture with HDR ON has a little over 900 draw calls.

Read over Unity’s HDR documentation and understand how it affects game performance. You should also know when it makes sense to use HDR to ensure you are receiving its full benefits.

Image Effects

Unity Pro comes with a range of image effects that enhance the look of a scene. Add Image Effects assets, even after creating your project, by going to Assets->Import Package->Image Effects. Once imported, there are two ways to add an effect to the camera. Click on your camera game object, then within the camera GUI, select Add Component, then Image Effects. You can also click on your camera object from the menu system by going to Component->Image Effect.

SSAO – Screen Space Ambient Occlusion

Screen space ambient occlusion (SSAO) is an image effect included in Unity Pro’s Image Effect package. Figure 16 shows the difference between a scene with SSAO off and on. The images look similar, but performance is markedly different. The scene without SSAO ran at 32 FPS and the scene with SSAO ran at 24 FPS, a 25% decrease.

Figure 16. A same level comparison with SSAO off (top) vs. SSAO on (bottom)

Be careful when adding image effects because they can negatively affect performance. For this document we only tested the SSAO image effect but expect to see similar results with the other image effects.

Occlusion Culling

Occlusion Culling disables object rendering not only outside of the camera’s clipping plane, but for objects hidden behind other objects as well. This is very beneficial for performance because it cuts back on the amount of information the computer needs to process, but setting up occlusion culling is not straightforward. Before you set up a scene for occlusion culling, you need to understand the terminology.

    Occluder– An object marked as an occluder acts as a barrier that prevents objects marked as occludees from being rendered.

    Occludee– Marking a game object as an occludee will tell Unity not to render the game object if blocked by an occluder.

For example, all of the objects inside a house could be tagged as occludees and the house could be tagged as an occluder. If a player stands outside of that house, all the objects inside marked as occludees will not be rendered. This saves CPU and GPU processing time.

Unity documents Occlusion Culling and its setup. You can find the link for setup information in the references section.

To show the performance gains from using Occlusion Culling, I set up a scene that had a single wall with highly complex meshed objects hidden behind. I took FPS captures of the scene while using Occlusion Culling and then without it. Figure 17 shows the scene with the different frame rates.

Figure 17. The image on the left has no Occlusion Culling so the scene takes extra time to render all the objects behind the wall resulting in an FPS of 31. The image on the right takes advantage of Occlusion Culling so the objects hidden behind the wall will be rendered resulting in an FPS of 126.

Occlusion culling requires developers to do a lot of manual setup. They need to also consider occlusion culling during game design as to make the game’s configuration easier and performance gains greater.

Level of Detail (LOD)

Level of Detail (LOD) allows multiple meshes to attach to a game object and provides the ability to switch between meshes the object uses based on camera distance. This can be beneficial for complex game objects that are really far away from the camera. The LOD can automatically simplify the mesh to compensate. To see how to use and setup LOD, check out Unity’s online documentation. The link to it is in the reference section.

To test the performance gains from LOD, I built a scene with a cluster of houses with 3 different meshes attached to them. While standing in the same place, I took an FPS capture of the houses when the most complex mesh was attached. I then modified the LOD distance so the next lesser mesh appeared, and took another FPS capture. I did this for the three mesh levels and recorded my findings as shown in Table 5.

Figures 18, 19, and 20 show the three varying levels of mesh complexity as well as the number of polygons and vertices associated with each mesh.

As I switched between the different LOD models, I took FPS captures for comparison (Table 7).

Table 7. LOD FPS comparison switching between lower model meshes

Table 7 shows the increased performance gains from setting up and using LOD. The FPS capture shows significant performance gains when using lower quality meshes. This however, can take a lot of extra work on the 3-D artists, who must produce multiple models. It is up to the game designer to decide whether or not spending the extra time for more models is worth the performance gains.

Batching

Having numerous draws calls can cause overhead on the CPU and slow performance. The more objects on the screen, the more draw calls to be made. Unity has a feature called Batching that combines game objects in to a single draw call. Static Batching affects static objects, and Dynamic Batching is for those that move. Dynamic Batching happens automatically, if all requirements are met (see batching documentation), whereas Static Batching needs to be created.

There are some requirements for getting the objects to draw together for both Dynamic and Static Batching, all of which are covered in Unity’s Batching document listed in the references section.

To test the performance gains of Static Batching, I set up a scene with complex airplane game objects (Figure 21) and took FPS captures of the airplanes both with batching and without batching (Table 8).

Figure 21. Static Batching Test scene filled with very complex airplane meshes

Table 8. Showing the difference between FPS and Draw Calls while turning static batching on and off for the test scene (Figure 21)

Unity’s batching mechanism comes in two forms, Dynamic and Static. To fully see the benefits from batching, plan to have as many objects as possible batched together for single draw calls. Refer to Unity’s batching documentation and know what qualifies an object for dynamic or static batching.

Conclusion

While Unity proves to be fairly simple to pick up and develop with, it can also be very easy to get yourself into performance trouble. Unity provides a number of tools and settings to help make games perform smoothly, but not all of them are as intuitive and easy to set up as others. Likewise, Unity has some settings that when turned on or used inappropriately can negatively affect game performance. An important part of developing with Unity is to have a plan before starting because some of the performance features require manual setup and can be much more challenging to implement if not planned at the project’s creation.

References

Quality Settings Documentation:
http://docs.unity3d.com/Documentation/Components/class-QualitySettings.html

Quality Settings Scripting API:
http://docs.unity3d.com/Documentation/ScriptReference/QualitySettings.html

Tech Demo Bootcamp:
http://u3d.as/content/unity-technologies/bootcamp/28W

Level of Detail Documentation:
http://docs.unity3d.com/Documentation/Manual/LevelOfDetail.html

Occlusion Culling Documentation:
http://docs.unity3d.com/Documentation/Manual/OcclusionCulling.html

Batching Documentation:
http://docs.unity3d.com/Documentation/Manual/DrawCallBatching.html

Rendering Path Documentation:
http://docs.unity3d.com/Documentation/Manual/RenderingPaths.html

Intel GPA:
http://software.intel.com/en-us/vcsource/tools/intel-gpa

Other Related Content and Resources

Unity MultiTouch Source (finally)
http://software.intel.com/en-us/blogs/2013/05/01/the-unity-multi-touch-source-finally

Implementing Multiple Touch Gestures Using Unity3D With Touchscript
http://software.intel.com/en-us/articles/implementing-multiple-touch-gestures-using-unity-3d-with-touchscript

Multithreading Perceptual Computing Applications in Unity3D
http://software.intel.com/en-us/blogs/2013/07/26/multithreading-perceptual-computing-applications-in-unity3d

Unity3D Touch GUI Widgets
http://software.intel.com/en-us/articles/unity-3d-touch-gui-widgets

About the Author

John Wesolowski, Intern
The focus of the group that I worked for at Intel was to enable Intel® chipsets for upcoming technology, with a focus on video games. It was our task to test the latest and upcoming video games to find potential bugs or areas of improvement inside the Intel® architecture or in the video game.

Outside of work, my all-time favorite activity used to be playing Halo* 2 online with my friends but since Microsoft shut down all Xbox LIVE* service for original Xbox* games, my friends and I like to LAN Halo 2 whenever we can. I also enjoy playing poker and flying kites. I am currently attending California State University, Monterey Bay and pursuing a degree in Computer Science and Information Technology.

Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.
Copyright © 2013 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.

Intel is excited to announce that we’re the platinum sponsors for this year’s 2014 D.I.C.E. Summit! Please stop by the D.I.C.E. Arcade area to catch a glimpse into both Intel and Havok’s latest gaming platform directions, technologies, and solutions. Intel is proud to promote this year’s event that will culminate with the 17th Annual D.I.C.E. Awards (Press Release) which will be streamed live at Feb 6th, beginning at 7:30 PM PST (Twitch Link Details)

Over seven hundred attendees will catch a glimpse of the several demos we’ll be showing in the D.I.C.E. Arcade lounge.  The 2 couch demo areas will showcase thin and light mobile Tablets and an Ultrabook™ playing Windows based PC games.  On a corner demo station we will be showing one of the Gigabyte™ BRIX Steam Machines playing SteamOS PC Gaming.  Havok will also join us with a Project Anarchy Demo; and an announcement of the University Program winners.  Other major highlights of the show will include a fund raiser, several dynamic speaking sessions; all followed by the D.I.C.E. Awards.   Here's a list of the big 3 highlights to focus in on:

• Intel:  Kick your feet up and join us in the Intel couch demo area to see the latest and greatest thin and light tablets and an Ultrabook™ with Iris Pro Graphics. In addition to this you won’t want to miss the Gigabyte™ BRIX Steam Machine Demo we’ll have on hand. Meet our experts to discuss these exciting products and what’s coming next.
• Havok:  Havok (You’ll want to get this verbiage from Kim/Kristin at Havok.  My guess is you’ll want to mention both the Project Anarchy; and the Univ chipshot also coming from Randi/Anna Cheng)
• D.I.C.E. Live Twitch Feed:  2014 D.I.C.E. Awards to be brought to you by Twitch!  Watch it live! Details here.

For D.I.C.E. Summit specifics please visit http://www.dicesummit.org/

See you all there!