Dev09 – la battaglia del touchless

DEV09 – LA BATTAGLIA DEL TOUCHLESSMatteo Valoriani

@MatteoValoriani

Clemente Giorio

@Tinux80

#CDays15 – Milano 24, 25 e 26 Marzo 2015

Grazie a

PlatinumSponsor

http://www.aspitalia.com/

http://www.aspitalia.com/

http://www.domusdotnet.org/

http://www.domusdotnet.org/

http://www.dotdotnet.org/

http://www.dotdotnet.org/

http://www.dotnetliguria.net/

http://www.dotnetliguria.net/

http://www.dotnetside.org/

http://www.dotnetside.org/

#CDays14 – Milano 25, 26 e 27 Febbraio 2014

Who we are…

Matteo ValorianiMicrosoft MVP Kinect for WindowsIntel Software InnovatorPhD on HCI @Politecnico di MilanoCEO of Fifth IngeniumConsultant

Clemente GiorioMicrosoft MVP Kinect for WindowsSenior Developer at Crionet srlInstructor at OvernetAuthor at Packt PublishingConsultant


Agenda

• HCI evolution

• Microsoft Kinect

• Intel Real Sense

• Leap Motion

• Duo3D

• Myo

• MaindWave

UI evolution

5Natural User Interface


You have to be a magician…

or at least a good illusionist

https://www.youtube.com/watch?v=u_diRgwPCS8

https://www.youtube.com/watch?v=u_diRgwPCS8


http://youtu.be/Hi5kMNfgDS4

http://youtu.be/Hi5kMNfgDS4


Kinect 2 - Specs

3D DEPTH SENSOR

RGB CAMERA

MULTI-ARRAY MIC

Hardware:

Depth resolution:512×424

RGB resolution:1920×1080 (16:9)

FrameRate:30 FPS

Latency:60 ms



Kinect 2 vs Kinect 1Version 1 Version 2

Depth range 0.4m → 4.0m 0.4m → 4.5m

Color stream 640×480 (VGA) 1920×1080 (Full HD)

Depth stream 320×240 @8bits 512×424 @16bits

Infrared stream None 512×424

Audio stream 4-mic array 16 kHz 4-mic array 48 kHz

USB 2.0 3.0

Hand Traking External tools Yes

Face Traking Yes Yes+Expressions

FOV 57° H 43° V 70° H 60° V

Tilt Motorized Manual


demo

Cons:The device is large

A dedicated power cord is required

The amount of data generated by the sensor also tends to saturate a USB controller

Kinect cannot easily distinguish individual fingers on a hand

Most of the features listed above require the Microsoft SDK, which is only supported for desktop applications on Windows 7 and 8.

Pros Vs. Cons

Pros:Skeletal trackingFace trackingMultiple sensors:Raw dataVoice control

Best for:Kiosks, installations, and digital signage projects where the user will be standing fairly far away from the display.



Understands 4 basic types of input

15Intel® RealSense™ Hands-On Lab - Milan

Categories of Input Capabilities Features

Hands • Hand and Finger Tracking • Gesture Recognition

• 22-point Hand and Finger Tracking• 9 static and dynamic mid-air gestures

Face • Face Detection and Tracking

• Multiple Face Detection and tracking• 78-point Landmark Detection (facial features)• Emotion Recognition (7 emotions, coming post-Beta)• Pulse Estimation• Face Recognition (Coming post-beta)

Speech • Speech Recognition • Command and Control• Dictation• Text to Speech

Environment • Segmentation• 3D Scanning• Augmented Reality

• Background Removal• 3D Object / Face / Room Scanning (Coming post-beta)• 2D/3D Object Tracking• Scene Perception (coming post-beta)

Understands Hardware Limits



demo

Cons:Getting some of the deeper features (like age and gender detection) to work is a bit tricky.

Device and software are in beta.

Due to the close range of the tracking system, hand gestures must be designed such that a user’s hand doesn’t occlude their own view of the display.

Pros Vs. Cons

Pros:

Smaller and less expensive

Close-range tracking

Hand posture/gesture recognition

Facial analysis

Speech (built-in support for speech synthesis powered by Nuance).

Raw data

Framework support: Processing, Unity and OpenFrameworks

Best for:Desktop/laptop applications where the user will be seated in front of the PC.Close range applications where features, apart from hand tracking and recognition, are necessary without too much precision or accuracy.


https://www.youtube.com/watch?v=_d6KuiuteIAhttps://airspace.leapmotion.com/

https://www.youtube.com/watch?v=_d6KuiuteIA

https://airspace.leapmotion.com/

Leap Motion

Leap Motion -Field of View

150° - Long Side

120° - Short Side

Max 60 cm abovethe controller

Max 60 cm wide on each side

Leap Motion for Developers


demo

Cons:Sensing range is fairly limited.

Only fingers are tracked.

There is no skeleton or face tracking.

No access to the raw sensor data.

Pros Vs. Cons

Pros:

Finger tracking is fast and accurate.HMD Mountable

Smaller and less expensive

Framework support: .NET, Processing, Cinder, etc.

Compatible: Mac OS and Windows

Best for:Controlled kiosk environments with a pointing-based UI.Generally best for general audience desktop apps which can be distributed in the Airspace store. Head-mounted display video pass-through. Augmented Reality. Computer Vision.


https://youtu.be/XNztZ2Chmzg

Cons:SDK SamplesDevice APIDense3D APIDocumentation

Pros Vs. Cons

Pros:

Ultra Compact Design (52x25x11mm).Fast (360 FPS @320x120, 56FPS @752x480).FoV 170° with Low Distortion ( < 3% )Acceleromet, gyroscope and temperature sensor.HMD MountableOutdoor with environmental IR.Cross Platform

Best for:Augmented Reality and Virtual Reality HeadSet. Outdoor, portable applications. Mird-range depth sensing with low latency.

Leap, RealSense, Kinect


2,5 cm 60 cm 2 m 4 m



demo

Cons:Calibration

Dry Eyes

Near range

Pros Vs. Cons

Pros:

Accuracy

Low Latency

Lenses and glasses support

Price

Best for:EyeTraking, Marketing Analysis, Medical device option



Hand Gestures


demo

Cons:Confort One device for each user.For complex scenarios required multiple devices

Pros Vs. Cons

Pros:WearableHand gesturesMulti Platform (Windows, Mac, iOS and Android)Unity support

Best for:Outdoor, portable applications.


demo

Cons:Confort One device for each userFew sensors

Pros Vs. Cons

Pros:WearableCheapHand gesturesMulti Platform (Windows, Mac, iOS and Android)

Best for:Outdoor, portable applications.


Recap

Some selection criterias…1° 2° 3° 4°

2 or + users Kinect 2 Intel Leap

Full body interaction Kinect 2 - -

Hand Gesture Recognition Myo Intel Leap Kinect 2

Accuracy Leap Intel Kinect 2 Myo

Voice command Intel Kinect 2

FaceTraking Tobii Intel Kinect

Commecial use Kinect 2 Intel Tobii Leap Myo

Compatibility Leap MYO Tobii Intel Kinect 2

Capture Volumes


The user is performing a hand gesture outside of the capture volume. The camera will not see this gesture

Evaluate different settings and environment


Sensor with Camera use IR light and Sunlight can blind the camera!!!

• Check exposition during all day

• Verify that there isn’t direct light on the camera

Indoor/Outdoor


These devices aren’t a Rugged devices:

• Check temperatures (+3/33°)

• Check humidity

Indoor/Outdoor (2)


Comfortable positions

Your users are not GORILLAS!!!

Intel® RealSense™ Hands-On Lab - Milan 45

User posture may affect design of a gesture


Input variability


Feedback, feedback, feedback,…


View of user:

• User Viewport

• User Overlay

… where actions performed for some other purpose or unconscious signs are interpreted in order to influence/improve/facilitate the actors' future interaction or day-to-day life (from Alan Dix)

• The interaction is not purposeful from the person side, but it is designed “to happen”

• It “happens” in relation to signs which are not done for that (body temperature, unconscious reactions such as blink rate, or unconscious aspects of activities such as typing rate, vocabulary shifts (e.g. modal verbs), actions done for other purposes, …

• It is designed for people acting

Manage Incidental Interaction



Q&A

Tutto il materiale di questa sessione suhttp://www.communitydays.it/

Lascia subito il feedback su questa sessione,potrai essere estratto per i nostri premi!

Seguici suTwitter @CommunityDaysITFacebook http://facebook.com/cdaysit#CDays15

http://www.communitydays.it/

Devices & Hardware

Dev09 – la battaglia del touchless