0 / Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation

1 /

Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.

Aerospace days

Airborne Vision systems for situation awareness

15/10/2014 Emmanuel KLINGOptronics & Defence DivisionSight Programs Department

2 /

Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.

Sagem A wide range of embedded Optronic Systems

CONFIDENTIEL / DATE / DIRECTION

I.R. SeekersAASM

Dismounted Soldier

UAVsAerosurveillance

AirborneOptronics

NavalOptronics

Land Optronics

Hand-heldOptronics

Thermal Imagers

3 /


Proprietary key technologies


Composants optroniques

Stabilisation & Localisation

Ligne de Visée

IntégrationOptomécanique

Optronic Sensors •Cooled and uncooled InfraRed•Any bandwidth•Lasers•ITAR Free Robust to bad weather conditions

Fog / low visibility

Real Time Video Processing•3D-Obstacle detection•Image Fusion•Enhanced reality•Tracking

Critical HW & SW•DAL level D to B•DO 178 B or C •DO254

Line of sight management•Stability•Geo location

Proved design•Airborne application standards•Robust / simple / reliable•Low weight

4 /


Observation

EUROFLIRTM : Gyro-stabilized Electro-Optical Airborne gimbals for- SPERWER Tactical UAS- PATROLLER UAS- NH90, PANTHER, COUGAR helicopters- Fixed-wings aircrafts

Targeting Helicopter sights

STRIX familiy (for TIGER HAP, HAD, ARH) Osiris (for TIGER UHT)

Situation Awareness To reduce the risk of accident during all phases of the flight

Mid Air Collisions between aircrafts CFIT (Controlled Flight into Terrain) Obstacle collisions on ground Additional Operational credit during landing phase All aircrafts : airliners, business jet, small aircraft, helicopters, UAs

Sagem Optronics in airborne applications

5 /


Some “Situation Awareness” devices in Avionics

Obstacle WarningSystems

LOASSWORDHELLAS

LOAM…

Collision AvoidanceSystems

Enhanced VisionSystems

Synthetic VisionSystems

Terrain Awareness& Warning Systems

6 /


Traffic Separation Rules

« In flight »

Approach

Helicopter Flight

7 /


Collision Avoidance products

TCAS – Transponders – PCAS – ADS-B TCAS (Traffic collision Avoidance Systems)

involves communication between all aircrafts equipped with an appropriate transponder

builds a three dimensional map of aircraft in the airspace Is only able to interact with aircraft that have a correctly operating mode

C or mode S transponder Elaborates avoidance tactics

PCAS (Portable Collision Avoidance System) Passive devices “hearing” transponder communications

ADS-B (Automatic dependent surveillance-broadcast) Active systems broadcasting aircraft current track, position and altitude

given by GPS to ATC and to other equipped ADS-B aircraft Elaborates avoidance tactics

Aircraft sensing by cooperative means Effective but requires that all aircraft in flight should be equipped with

TCAS / ADS-B High Certification cost (TCAS & transponders) PCAS & ADS-B not yet certified worldwide Fixed and limited channel data bandwidth (ADS-B)

8 /


Mid Air UAV and helicopters Operates mainly in uncontrolled airspaces TCAS and transponders are not well designed for helicopters since they are not installed on

every following aircrafts helicopters gliders ULM UAV Balloons Paragliders

We propose to increase the capability in traffic separation with a Traffic Sensing product using non-cooperative and independent means

Independant The aircraft is not dependent on any external element (eg GPS) to make traffic surveillance

in its environment.

Non-cooperative Traffic monitoring is done without dedicated radio emission provided by other aircrafts.

Therefore, it applies to all types of aircraft.

The need of a non cooperative and independent traffic surveillance system

9 /


TAWS (Terrain Awareness and Warning System) Electronics coupled to the navigation system and to a Digital cartographic and

terrain database Developed since the 2000s, initially for military aviation Also for airliners and business aircraft, and now for helicopters Returns alerts about CFIT if the extrapolated trajectory of the plane "cuts" the ground

SVS - Synthetic Vision System synthetic Image generation on the basis of terrain and cartographic database, and

according to the trajectory of the aircraft Eventually merge with the images provided by an EVS, Becomes a CVS

(Combined Vision System) CVS and SVS are subject to regulations set by D0315 (aircraft only). There is no equivalent regulation for such equipment on board helicopters.

On airplanes, SVS shall display obstacles which are higher than 200 feet …

(DO309)Helicopters operate in widely diverse environment, including under visual and instrument flight rules (VFR and IFR) and in both on- and off-airport operations, often with significant low altitude exposure.

These systems seem not well-suited to helicopter operations since the used database is DTED1 or lower (100 m cell size) and not robust to ground asperities

Controlled Flight Into Terrain - Existing equipments

10 /


HTAWS Regulation (RTCA DO 309)

Position Source Some recommendations (no requirements) regarding the type of GPS or WAAS used. If the position source is corrupted, the HTAWS must be switch off

Terrain and Obstacle database No requirement on the quality of the database or its accuracy, nor on the recurrence of its update “The highest resolution database should be used where necessary and available.”(DO309) “The HTAWS does NOT guarantee successful recovery from a conflict due to factors such as pilot response, aircraft

performance and database limitation” (DO309)

Typical Terrain Database accuracies For a DTED level 1 :

Individual Cell size : 100m Horizontal Accuracy r = (x2+y2)1/2 < 50 m @90% Altitude Accuracy z < 30 m @90%

It appears that: Only a database with metric accuracy allows to show the ground surface and generate alerts on small obstacles

(buildings, rocks, pylons…) in low or very low altitude High resolution Databases (metric accuracy) are often classified and localized mainly around strategic sites (airports,

military bases) The accuracy of the navigation data and spatial accuracy of the database must be compatible and constistent.

Therefore, the operational interest of HTAWS is still relatively low compared to the challenge of helicopter flight => the need of a system for detecting obstacles on real measurements: these systems are the OWS (Obstacle Warning Systems)

11 /


Enhanced Flight Vision Systems

Situational awareness

Approach stability in low visibility conditions

12 /


DO315 - Glossary

RVR : Runway Visual Reference ,in aviation meteorology, is the distance over which a pilot of an

aircraft on the centreline of the runway can see the runway surface markings delineating the runway or identifying its centre line. RVR is normally expressed in feet or meters.

« Enhanced Flight Visibility » : “average forward horizontal distance, from the cockpit of an aircraft in flight, at which prominent

topographical objects may be clearly distinguished and identified by day or night by a pilot using an enhanced flight vision system

-> An EFVS, then, is the means by which the pilot meets the enhanced flight visibility requirement

THRE : THReshold Elevation

TDZ : TouchDown Zone


13 /


Visual References

Approach light system, if installed;

or

Visual references: Runway threshold, identified by at least one of the following:

Beginning of the runway landing surface, Threshold lights, or Runway end identifier

lights

AND Touchdown zone, identified by at least one of the following:

Runway touchdown zone landing surface, Touchdown zone lights, Touchdown zone markings, or Runway

lights.


14 /


Active Obstacle Warning Systems for low altitude flights

AOWS – Active Obstacle Warning System Principle :

Scanning the space by a RADAR or LASER beam Range measurement allows to detect possible

obstacles.

15 /


Active OWS Example (ASC Inc.)

3D Flash LADAR Helicopter Landing Sensor for Brownout and Reduced Visual Cue


16 /


Active Obstacle Warning Systems for low altitude flights

Some products Lord / Sword (ELOP) : FO LASER, 24 kg, 600w Eagle Owl (BAe Systems) : FO LASER, 20 kg LOAM (Finmeccanica) : FO LASER, 25 kg LOAS (Goodrich) : FO LASER, 15 kg OAsys (Amphitec) : RADAR FMCW (Ka band) HELLAS (Airbus Defence) : FO LASER, 25 kg

Typical sell prices : 150 k$ - 300 k$

The OWS provide a real answer to the problem of obstacles in the flight at very low altitude, especially on cable detection.

Like all active systems, this capability shall not be at the expense of human safety -> need to switch off active systems during final approach to preserve personal safety (LASER) or electronic devices (RADAR).

Cost, size, consumption and the lack of any certification rules associated with these products are some reasons why this market is reserved to military markets

17 /


POSAS Concept : A passive optronic situation awareness system dedicated to UAS and helicopters

to offer standard EFVS operational credit in approach phase (with Runway Visual References Recognition)

to strengthen the traffic separation capability using non-cooperative and independent means in addition to cooperative means

to offer a credible, passive, compact and low cost alternative compared to current active obstacle warning systems based on LASER or RADAR

to provide solutions to extend TAWS and SVS operational credit at low altitude.

POSASWide FOV EVS

Passive OWS

Traffic Surveillance

3D Mapping

18 /


POSAS Concept : How should it work ?

Integration of distributed low cost optronic sensors

Uncooled infrared sensors operating in the LWIR spectral band Day and night “hot spot” detection

Low light level sensors operating in the visible and near infrared spectral bands Aircraft strobe ligths, helipad lights, buildings warning lights…

Integration of advanced image processing technologies

Stereovision & Simultaneous Localisation and Mapping 3D Terrain mapping Navigation data accuracies improvement

Hot spot detection Algorithms Reminder : tracks with a constant bearing are on a collision trajectory

19 /


Mid Air collision avoidance system

MIDCAS Programme (AED contract, 2008) :

UAV insertion in the traffic

Wide selection of sensors / devices to be tested : Distributed Uncooled Infrared Sensors (Sagem) Near Infrared Sensors (Diehl BGT Defence) RADAR (THALES) Cooperative devices : TCAS, ADS-b

Sensors installed on a Sky-Y UAV (Alenia Aeronautica) and DGA EV testbed (CASA Aircraft)

Sensor Head Uncooled LWIR micro camera DGA EV Testbed

20 /


POWS Demonstrator

Application : Flight Safety for Rotary Wings in degraded visual environment

Flight cleared demonstrator currently under test 6 uncooled Infrared Cameras 1 visible HD camera

Wide area surveillance : 150°x 80°

3D StereoVision

Automatic obstacle detection

Fly test on a DGA EV helicopter Test Bed (PUMA)

21 /


POWS Demonstrator

22 /


STEREOVISION

Goal: compute a dense distance map (one distance for each pixel) from a pair of images

Prerequisite: need to know precisely the geometry of the stereo rig (focal, pitch, baseline, etc.)

Steps: Rectification: adjust the images as if they were co-

planar (linear operation) Block matching: for each pixel of the left image, find its

correspondent in the right image Triangulation: compute the distance map (meters) from

the disparity map (pixel)

Rectified left image

Left image

Rectification

Rectified right image

Right image

Rectification

Block matching

Disparity map (pixel)

Triangulation

Distance map (m)

23 /


Simultaneous Localisation and Mapping

Associated data

Images

Low level extraction

3D non-linear filtering

High level

Interest point detection(Harris, SIFT, Kanade…)

Temporal association

INS/Vision hybridation

Tracks(3D over time)

24 /


Enhanced Vision (Wide FOV)

Obstacles detection and ground mapping (3D)

Obstacle Detection performances Between 0,6 NM and 10 NM (according to object type and meteorological conditions)

Localisation and ranging performances Angular accuracy (azimut, elevation) : < 1mrad @ 3 Range Accuracy : < 1% @ 3up to 1Mile

POSAS : Main Operational Capabilities

Detection & ranging TAWS AOWS POWSTerrain (ground) Hills Structures (buildings) Pylons Câbles ?

Oil rigs Ships Street lights Trees Helipad

25 /


POSAS : Additional Operational Capabilities

Providing of a “USER” obstacle and terrain database for HTAWS and SVS Metric-class accuracy (obstacle location) 3D terrain mapping on a corridor along the path of the helicopter with a width of about 1 km Terrain Aided Navigation : the software compares the measured 3D mapping of the terrain with

its internal database. The correlation peak gives the position of the aircraft in geographic coordinates.

Improvement of navigation Data accuracies by INS / Vision hybridization Compared to the INS alone, the gains expected by hybridization with vision are:

Attitude: error reduction of 35% Position: error reduction of 90% Speed: error reduction of 85% Strong gains are expected in z (better than GPS)

Average standard error in z: 30 cm! (between 5cm and 2m on a typical scenario)

26 /


OBSTACLE WARNING SYSTEMARCHITECTURE À L’ÉTUDE

AOWSEV_SV Fusion

Symb Processor

DEUs

Terrain Database Generator

and SV ProcessorFlight Control

Computers

ACFT Geo PositionMAGRS/EGI (GPS)

I2HMD (COTS/GDE)

DTED 2+ MSD

DIRECT

HMD Tracked FOR - Display

I2HMDProc

HMD Head Tracker

DTED/HRTI Update

Synthetic Vision Data

DigMap Dig

Map

I2HMDProc

HMD Head Tracker

POWS

1553/Ethernet

1553/EthernetCPLD

G RO WTH

GR

OW

TH

A U T O N A C

HVR CPLD REM HO VER

O N

A R M

ACT

CPLD

I NC

DECR

VS LM TCRS HDG

HO LD

SPEED

HO LD

ALT

HO LD

SRCH PAT WYPTAPPR

E

N

A

V

I

N

A

V

TACANVO R/ I LSG RO WTH

GR

OW

TH G R

BRTA R M

ACT

A R M

ACT

A R M

ACT

A R M

ACT

A R M

ACT

A R M

ACT

O NO NO N

P U S H

X 1 0

O NO N

CO

NT

RO

L P

AN

EL

AR

EA

PE

R

PH

OT

O E

TC

H

DR

AW

ING

S E L

T

F A IL

O F F

O N

B R T

12XXX

USABLE FUELFEED

XXXXX

FLAPSENG

TRQ

NR

XXX

NP

XXX

XXX

90

HYD 2

FAI LHYD 3

FAI L

HYD 1

FAI L

EST EST

XXX

NPNP

XXX

FEED

EST

XX XX

A LT

01

3

45

6

7

8

9

9

99 600

6 08 0

1 0 0

1 2 0

1 4 0

1 6 0

2 0 0

1 8 02 5 0

3 0 0

K N O T S

IA S

3 5 0

Y

2 3

5 6

8 9

A B

H

O P

C D

J K

Q R

W X

E F

L M

S T

Z

1

4

G

N

U

7

V

/ 0

IN A V E N A V

F P L N WY P T E C S

IF F M S N W/W

R T N

E M C O N

C O M M

D E L C L R

T IM E R

E M E R

L T S

E N T E R

ID E N T

P G

P G

S E Q

S E Q

Y

2 3

5 6

8 9

A B

H I

OP

C D

J K

Q R

W X

E F

L M

ST

Z

1

4

G

N

U

7

V

/ 0

IN A V E N A V

F P L N WY P T E C S

IF F M S N W/W

R T N

E M C O N

C O M M

D E L C L R

T IM E R

E M E R

L T S

E N T E R

ID E N T

P G

P G

S E Q

S E Q

C HC H

BR

T

C HAN RFILC R STB Y SECACT I VES

E

L

C HC H

BR

T

C HAN RFILC R STB Y SECACT IVES

E

L

C D U A C K C D US Y S

S T A TC /AE IC A S

C A C

B R T

O F F

N IG H T

D A Y

C A C

CO M M 1

01N/ 251. 000

CO M M 1

13 N/ 123. 50

VO R/ I LS FREQ

110. 30

TCN CHAN

123X

BARO PRES

29. 92RAD ALT

LO W SET

200

M O DE 3 CO DE

7600C

X X X X

M G T

X X X X

N G

X X XX X X X X X

X X X

X X X

X X X XX X X XX X X X

XXX XXX

X X X

P S I

X X X

O IL

P S I

P R G B

T A G B

P S I

C

H Y D 1 H Y D 2 H Y D 3

P S I

C

C

XXX

XXX XXX

XXX

XXXO

O

O

APU O N

EM CO N

RAM P O PEN

M WG B

DC BUS 1 FAI L

AFT CRO WN O PEN

HY D SYS 1 PRESS HI

L FEED TANK LO

RADAR BEACO N FAI L

HY D SYS 1 PRESS HI

HY D SYS 1 PRESS HI

CO M M 1

01N/ 251. 000

CO M M 1

13 N/ 123. 50

VO R/ I LS FREQ

110. 30

TCN CHAN

123X

BARO PRES

29. 92

RAD ALT

LO W SET

200M O DE 3 CO DE

7600C

PFD NAV FLI R STAT

AVI O NI CS PO WER SELECT

SYST*

AVI O NI CS EQ UI PM ENT PO WER

PO WER

O FF

PWR

ALL

O N

SEQ

O FF

O N

O N

O N

O N

O N

O N

O N

O N

O N

O N

O N

ADF

ARC 210 #2

DEU - CO PI LO T

DM S

FM HO M I NG

I FF

I NS 1

I NS 2

I NS 3

RAD ALT

SATCO M

TACAN

S Y M

C O N B R T

A C KC /A

O F F

N IG H T

D A Y

S Y M

C O N B R T

A C KC /A

O F F

N IG H T

D A Y

S Y M

C O N B R T

A C KC /A

O F F

N IG H T

D A Y

S Y M

C O N B R T

A C KC /A

O F F

N IG H T

D A Y

S L E W

P R E S S D E S IG N A T E

S L O W

F A S TN F O V W

P O L

HDD HDDDisplay System (CMS)

1553/Ethernet

DIRECT

HMD Tracked FOR - Display

DEUs

Current EquipmentAugmenting Equipment

Optional Equipment

DEUs : Display Electronic UnitsDTED : Digital Terrain Elevation DataEV : Enhanced VisionSV : Synthetic VisionHMD : Helmet Mounted DisplayCMS : Cockpit Management SystemDAS : Distributed Aperture SystemHRTI : High Resolution Terrain InformationPOWS : Passive Obstacle warning SystemAOWS : Active Obstacle Warning SystemHDD : Head Down Display

DVE SYSTEM ARCHITECTURE (NIAG SG 167)

DAS Uncooled-IR / LLLTV

27 /


Conclusion

A Passive Optronic System to increase Situation Awareness

to strengthen the traffic separation capability using non-cooperative and independent means

OK, by using distributed staring sensors and air surveillance algorithms

to offer a credible, compact and low cost alternative compared to current obstacle warning systems based on LASER or RADAR

OK, by using distributed staring sensors and SLAM/Stereovision algorithms, the system can provide alerts when detecting obstacles on ground

to provide solutions to extend HTAWS and SVS operational credit at low altitude.

OK, by using SLAM / stereovision Algorithms : the system may elaborate a “user” 3D mapping of the neighborhood and may improve the accuracies of the navigation data (INS/Vision hybridization, Terrain Aided Navigation)

to provide day / night enhanced vision for HDD, HuD, Helmet Mounted Display…

28 /


Questions ?

Documents

0 / Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation