Interactive Touch Screens *Working Demo*

Game Engine Game Engine Support and Discussion
#1

Ok, heres a big one…

Does anyone have any idea of where to start to make the kind of touch screens seen in Doom 3 etc. (or perhaps not that complex, but a more simple version)

Ive had a look around, not only for BGE stuff, but any python game tutorials on the topic, but haven’t come up with anything.
Heres an example image, from Doom 3, incase some people haven’t played the game.

Doom3
http://img166.imageshack.us/img166/3730/doom3screentz1.jpg

Blender 3D
http://img58.imageshack.us/img58/4212/screencr2.jpg

Basically by walking a certain distance from the screen you take control of the cursor, and can press the buttons etc and interact with the screen.
Im not so fussy on the cursor in the screen bit, because that would be very difficult.

But if it was possible to press a button using a visible cursor in the GE, using rasterizer.showMouse() to see the mouse, that would be good. I guess id have to define a certain area of the computer screen which could be clicked on, and somehow animate the onscreen button in that area when the cursor is over it (so the player can see which button their hovering over).

Btw im not asking someone to do this for me, id like to learn how myself, i just need some ideas on where to start etc, because i havent been able to come up with anything at all. And a few tests ive tried in the GE (non-python) haven’t worked out at all.

Ps, could the new GLSL features make this a more possible ‘device’ for the GE?

Update Got it working pretty well, thanks to everyones help, the latest .blend is here - http://www.box.net/shared/97xx1ce8t4

#2

each botton is a diff plane with a mouse over and mouse click sensor conneted to a and controller the actuator not sure.

#3

Yeh, that was what i tried out first. But theres a few issues with it, mainly the fact that you have to have a ‘button plane’ really close to the screen, and blender seems to have issues working out which one is on top… It tends to flicker between the 2. Ive tryed it at a whole lot of different distances, and it still doesnt work properly.

It would be a really nice and easy way to do it though…

#4

why dont you do what most other games do and use a different scene that mimiks the view in the ‘live game’ that puts you directly in front of the console without movement controls until you press {escape} or something like that. That way its just a 2d area mapping exercise.

#5

Thats a good idea, but i dont think it would work the way i need for this…

For example, if your flying a ship which is controlled by a bunch of touch screens, you wouldnt want to exit to the screen every time you needed to adjust the engines or something. You’d want to just be able to look down at the screen, press the appropriate button, and then look straight back up again. Thats kinda the effect im looking for…

Perhaps theres some way to get the computer screen scene, and map it to a plane. Or something. Sounds complicated though… Thats why i was thinkning that GLSL could be handy, because i dont think it would be possible (at least without years worth of coding) using the older GE.

Thanks for the suggestion though, ill have to think about using that technique if i cant work anything else out.

#6

Hmm… I would make a bunch of empties on the screen and have it add planes when you need them. And delete them when you need to. Then use a “screen” on it which is one plane and use replace mesh (because it replaces the ME which contains the materials)

I hope this made enough sense!

#7

i would just make the screen a plane, and have invisible planes over each button that you press. also, for the mouse, if you use either a near sensor or getDistanceTo(), you could determine when to put r.showMouse(1).

#8

to make the mouse on the screen only can’t you set the frame posi and frame size to the posi and size of the monitor.

#9

oh come on guys this is EASY! heres the blend file: touch-screen.blend (149 KB)

#10

Yeh that sounds good, but it doesnt let you animate the buttons to ‘glow’ etc.

Thanks for that .blend, its pretty helpful.

The cursor issue is no longer a problem, but, the text/buttons on planes hovering above the screen are.

Heres a simple .blend, with all the work up to date. (including the use lipsonfire’s script)
http://www.box.net/shared/30ghpqbph8

You can use W,A,S,D to move around, and the mouse to look around (pretty basic set up). Press the blue button on the side of the screen to turn it on. When the blue screen appears (and button goes red) you know the screen is on.
The issue should be easy to see, as before you turn on the screen the ‘welcome’ text can still be seen, overlapping the black screen occasionally.

It does this for textured planes as well, and is pretty much the only thing stopping all this from working…

#11

That problem is easily fixed, use a visibility actuator or the setVisible() method in python.

But gah, that looks nice.

#12

Lol thanks, its a pretty simple setup. I made it all in about 3 hours to test this kind of stuff out for an upcoming game, so i didn’t want to waste time on graphics etc.

And yes, the set visibility will work very well. I’ll work on it some more when i get home from work tonight. I think i have it all sorted out now. :yes:

#13

I didn’t take the time to dissect it, but I would try using a ray going forward from the screen/player. This way, when the player looks away from the screen, he doesn’t still have a mouse.

Also, I don’t know if this was just an issue with me using Linux, but the mouse was acting up. For example, it seemed kinda jittery and had ghosting issues.

Still pretty interesting.

Edit: I started looking at it, and I have another recommendation. Try getting rid of the near sensor if possible. These things have a tendency to eat the frame rate. Also 1 minute of fiddling with a ray sensor didn’t yield many results.

#14

Id be interested in knowing if anyone else has experienced this. It doesnt seem very smooth on my computer either.

Yeh i messed around with the ray sensor idea as well, but also couldnt get it right. It seems like a simple thing to do, but nothing seems to work.

And im aware of the near sensor, theres also one on the button object. I might look into another method of doing it. But with all the logic fixes and apricot stuff going on, im not to worryed about it at this stage…

#15

Yea the mouse is jittery, but thats because of the mouse look script you were using. It keeps putting the mouse’s x and y co-ordinates at (0,0), so when ever the mouse is moved it would jump back to the centre.

#16

Thanks andrew for the info, ive tried to dampen the resetting of the cursor, but haven’t been able to yet.

Also, lots of advancements with the touch screen. Its pretty much working exactly how ive been wanting.

Latest version: http://www.box.net/shared/umvr719fry
Turn the screen on/off with the blue/red button. And use the screen buttons to generate objects in the large tubes.

I also added a orientation constraint to the camera, which fixes the way it can rotate all the way up and all the way down, looping around and around. It works very well now! :cool:

2.47 is very nice…

#17

Another smallish update, got a second screen working. Im trying to get the workflow as simple as possible, and so it was very easy to set up a second screen, took about 5mins.

http://www.box.net/shared/97xx1ce8t4

The second screen opens and closes a door, which leads to a new room, where im going to start on a new problem i need to work out. This will probubly be the last update, mainly because ive got the screens working how i want.

If anyone can manage to get the cursor to only appear when the mouse is over the screen, that would be awesome. (currently you can turn around and look away from the screen, and the cursor still appears) Ray doesn’t seem to work very well, for some strange reason…

#18

I found another bug… When the screen is on, and you walk away from it and face it, you can still press the buttons. :frowning:

#19

Ah, yes, thanks for making me aware of that. Must have overlooked that bit.

I guess i could add a near sensor to all the buttons, but that would be bad… I think adding a property to the screen, which is true when the player is close, and all the buttons could copy this property, to see if it was true.

A little more work needs to be done still then… Ill work it out, when i have some time.

#20

What about an invisible plane, if the player collides with it then they must be far away from the screen. And you can run whatever script to remove the cursor?