1 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Aerospace days
Airborne Vision systems for situation awareness
15/10/2014 Emmanuel KLINGOptronics & Defence DivisionSight Programs Department
2 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Sagem A wide range of embedded Optronic Systems
CONFIDENTIEL / DATE / DIRECTION
I.R. SeekersAASM
Dismounted Soldier
UAVsAerosurveillance
AirborneOptronics
NavalOptronics
Land Optronics
Hand-heldOptronics
Thermal Imagers
3 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Proprietary key technologies
CONFIDENTIEL / DATE / DIRECTION
Composants optroniques
Stabilisation & Localisation
Ligne de Visée
IntégrationOptomécanique
Optronic Sensors •Cooled and uncooled InfraRed•Any bandwidth•Lasers•ITAR Free Robust to bad weather conditions
Fog / low visibility
Real Time Video Processing•3D-Obstacle detection•Image Fusion•Enhanced reality•Tracking
Critical HW & SW•DAL level D to B•DO 178 B or C •DO254
Line of sight management•Stability•Geo location
Proved design•Airborne application standards•Robust / simple / reliable•Low weight
4 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Observation
EUROFLIRTM : Gyro-stabilized Electro-Optical Airborne gimbals for- SPERWER Tactical UAS- PATROLLER UAS- NH90, PANTHER, COUGAR helicopters- Fixed-wings aircrafts
Targeting Helicopter sights
STRIX familiy (for TIGER HAP, HAD, ARH) Osiris (for TIGER UHT)
Situation Awareness To reduce the risk of accident during all phases of the flight
Mid Air Collisions between aircrafts CFIT (Controlled Flight into Terrain) Obstacle collisions on ground Additional Operational credit during landing phase All aircrafts : airliners, business jet, small aircraft, helicopters, UAs
Sagem Optronics in airborne applications
5 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Some “Situation Awareness” devices in Avionics
Obstacle WarningSystems
LOASSWORDHELLAS
LOAM…
Collision AvoidanceSystems
Enhanced VisionSystems
Synthetic VisionSystems
Terrain Awareness& Warning Systems
6 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Traffic Separation Rules
« In flight »
Approach
Helicopter Flight
7 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Collision Avoidance products
TCAS – Transponders – PCAS – ADS-B TCAS (Traffic collision Avoidance Systems)
involves communication between all aircrafts equipped with an appropriate transponder
builds a three dimensional map of aircraft in the airspace Is only able to interact with aircraft that have a correctly operating mode
C or mode S transponder Elaborates avoidance tactics
PCAS (Portable Collision Avoidance System) Passive devices “hearing” transponder communications
ADS-B (Automatic dependent surveillance-broadcast) Active systems broadcasting aircraft current track, position and altitude
given by GPS to ATC and to other equipped ADS-B aircraft Elaborates avoidance tactics
Aircraft sensing by cooperative means Effective but requires that all aircraft in flight should be equipped with
TCAS / ADS-B High Certification cost (TCAS & transponders) PCAS & ADS-B not yet certified worldwide Fixed and limited channel data bandwidth (ADS-B)
8 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Mid Air UAV and helicopters Operates mainly in uncontrolled airspaces TCAS and transponders are not well designed for helicopters since they are not installed on
every following aircrafts helicopters gliders ULM UAV Balloons Paragliders
We propose to increase the capability in traffic separation with a Traffic Sensing product using non-cooperative and independent means
Independant The aircraft is not dependent on any external element (eg GPS) to make traffic surveillance
in its environment.
Non-cooperative Traffic monitoring is done without dedicated radio emission provided by other aircrafts.
Therefore, it applies to all types of aircraft.
The need of a non cooperative and independent traffic surveillance system
9 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
TAWS (Terrain Awareness and Warning System) Electronics coupled to the navigation system and to a Digital cartographic and
terrain database Developed since the 2000s, initially for military aviation Also for airliners and business aircraft, and now for helicopters Returns alerts about CFIT if the extrapolated trajectory of the plane "cuts" the ground
SVS - Synthetic Vision System synthetic Image generation on the basis of terrain and cartographic database, and
according to the trajectory of the aircraft Eventually merge with the images provided by an EVS, Becomes a CVS
(Combined Vision System) CVS and SVS are subject to regulations set by D0315 (aircraft only). There is no equivalent regulation for such equipment on board helicopters.
On airplanes, SVS shall display obstacles which are higher than 200 feet …
(DO309)Helicopters operate in widely diverse environment, including under visual and instrument flight rules (VFR and IFR) and in both on- and off-airport operations, often with significant low altitude exposure.
These systems seem not well-suited to helicopter operations since the used database is DTED1 or lower (100 m cell size) and not robust to ground asperities
Controlled Flight Into Terrain - Existing equipments
10 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
HTAWS Regulation (RTCA DO 309)
Position Source Some recommendations (no requirements) regarding the type of GPS or WAAS used. If the position source is corrupted, the HTAWS must be switch off
Terrain and Obstacle database No requirement on the quality of the database or its accuracy, nor on the recurrence of its update “The highest resolution database should be used where necessary and available.”(DO309) “The HTAWS does NOT guarantee successful recovery from a conflict due to factors such as pilot response, aircraft
performance and database limitation” (DO309)
Typical Terrain Database accuracies For a DTED level 1 :
Individual Cell size : 100m Horizontal Accuracy r = (x2+y2)1/2 < 50 m @90% Altitude Accuracy z < 30 m @90%
It appears that: Only a database with metric accuracy allows to show the ground surface and generate alerts on small obstacles
(buildings, rocks, pylons…) in low or very low altitude High resolution Databases (metric accuracy) are often classified and localized mainly around strategic sites (airports,
military bases) The accuracy of the navigation data and spatial accuracy of the database must be compatible and constistent.
Therefore, the operational interest of HTAWS is still relatively low compared to the challenge of helicopter flight => the need of a system for detecting obstacles on real measurements: these systems are the OWS (Obstacle Warning Systems)
11 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Enhanced Flight Vision Systems
Situational awareness
Approach stability in low visibility conditions
12 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
DO315 - Glossary
RVR : Runway Visual Reference ,in aviation meteorology, is the distance over which a pilot of an
aircraft on the centreline of the runway can see the runway surface markings delineating the runway or identifying its centre line. RVR is normally expressed in feet or meters.
« Enhanced Flight Visibility » : “average forward horizontal distance, from the cockpit of an aircraft in flight, at which prominent
topographical objects may be clearly distinguished and identified by day or night by a pilot using an enhanced flight vision system
-> An EFVS, then, is the means by which the pilot meets the enhanced flight visibility requirement
THRE : THReshold Elevation
TDZ : TouchDown Zone
CONFIDENTIEL / DATE / DIRECTION
13 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Visual References
Approach light system, if installed;
or
Visual references: Runway threshold, identified by at least one of the following:
Beginning of the runway landing surface, Threshold lights, or Runway end identifier
lights
AND Touchdown zone, identified by at least one of the following:
Runway touchdown zone landing surface, Touchdown zone lights, Touchdown zone markings, or Runway
lights.
CONFIDENTIEL / DATE / DIRECTION
14 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Active Obstacle Warning Systems for low altitude flights
AOWS – Active Obstacle Warning System Principle :
Scanning the space by a RADAR or LASER beam Range measurement allows to detect possible
obstacles.
15 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Active OWS Example (ASC Inc.)
3D Flash LADAR Helicopter Landing Sensor for Brownout and Reduced Visual Cue
CONFIDENTIEL / DATE / DIRECTION
16 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Active Obstacle Warning Systems for low altitude flights
Some products Lord / Sword (ELOP) : FO LASER, 24 kg, 600w Eagle Owl (BAe Systems) : FO LASER, 20 kg LOAM (Finmeccanica) : FO LASER, 25 kg LOAS (Goodrich) : FO LASER, 15 kg OAsys (Amphitec) : RADAR FMCW (Ka band) HELLAS (Airbus Defence) : FO LASER, 25 kg
Typical sell prices : 150 k$ - 300 k$
The OWS provide a real answer to the problem of obstacles in the flight at very low altitude, especially on cable detection.
Like all active systems, this capability shall not be at the expense of human safety -> need to switch off active systems during final approach to preserve personal safety (LASER) or electronic devices (RADAR).
Cost, size, consumption and the lack of any certification rules associated with these products are some reasons why this market is reserved to military markets
17 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
POSAS Concept : A passive optronic situation awareness system dedicated to UAS and helicopters
to offer standard EFVS operational credit in approach phase (with Runway Visual References Recognition)
to strengthen the traffic separation capability using non-cooperative and independent means in addition to cooperative means
to offer a credible, passive, compact and low cost alternative compared to current active obstacle warning systems based on LASER or RADAR
to provide solutions to extend TAWS and SVS operational credit at low altitude.
POSASWide FOV EVS
Passive OWS
Traffic Surveillance
3D Mapping
18 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
POSAS Concept : How should it work ?
Integration of distributed low cost optronic sensors
Uncooled infrared sensors operating in the LWIR spectral band Day and night “hot spot” detection
Low light level sensors operating in the visible and near infrared spectral bands Aircraft strobe ligths, helipad lights, buildings warning lights…
Integration of advanced image processing technologies
Stereovision & Simultaneous Localisation and Mapping 3D Terrain mapping Navigation data accuracies improvement
Hot spot detection Algorithms Reminder : tracks with a constant bearing are on a collision trajectory
19 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Mid Air collision avoidance system
MIDCAS Programme (AED contract, 2008) :
UAV insertion in the traffic
Wide selection of sensors / devices to be tested : Distributed Uncooled Infrared Sensors (Sagem) Near Infrared Sensors (Diehl BGT Defence) RADAR (THALES) Cooperative devices : TCAS, ADS-b
Sensors installed on a Sky-Y UAV (Alenia Aeronautica) and DGA EV testbed (CASA Aircraft)
Sensor Head Uncooled LWIR micro camera DGA EV Testbed
20 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
POWS Demonstrator
Application : Flight Safety for Rotary Wings in degraded visual environment
Flight cleared demonstrator currently under test 6 uncooled Infrared Cameras 1 visible HD camera
Wide area surveillance : 150°x 80°
3D StereoVision
Automatic obstacle detection
Fly test on a DGA EV helicopter Test Bed (PUMA)
21 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
POWS Demonstrator
22 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
STEREOVISION
Goal: compute a dense distance map (one distance for each pixel) from a pair of images
Prerequisite: need to know precisely the geometry of the stereo rig (focal, pitch, baseline, etc.)
Steps: Rectification: adjust the images as if they were co-
planar (linear operation) Block matching: for each pixel of the left image, find its
correspondent in the right image Triangulation: compute the distance map (meters) from
the disparity map (pixel)
Rectified left image
Left image
Rectification
Rectified right image
Right image
Rectification
Block matching
Disparity map (pixel)
Triangulation
Distance map (m)
23 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Simultaneous Localisation and Mapping
Associated data
Images
Low level extraction
3D non-linear filtering
High level
Interest point detection(Harris, SIFT, Kanade…)
Temporal association
INS/Vision hybridation
Tracks(3D over time)
24 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Enhanced Vision (Wide FOV)
Obstacles detection and ground mapping (3D)
Obstacle Detection performances Between 0,6 NM and 10 NM (according to object type and meteorological conditions)
Localisation and ranging performances Angular accuracy (azimut, elevation) : < 1mrad @ 3 Range Accuracy : < 1% @ 3up to 1Mile
POSAS : Main Operational Capabilities
Detection & ranging TAWS AOWS POWSTerrain (ground) Hills Structures (buildings) Pylons Câbles ?
Oil rigs Ships Street lights Trees Helipad
25 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
POSAS : Additional Operational Capabilities
Providing of a “USER” obstacle and terrain database for HTAWS and SVS Metric-class accuracy (obstacle location) 3D terrain mapping on a corridor along the path of the helicopter with a width of about 1 km Terrain Aided Navigation : the software compares the measured 3D mapping of the terrain with
its internal database. The correlation peak gives the position of the aircraft in geographic coordinates.
Improvement of navigation Data accuracies by INS / Vision hybridization Compared to the INS alone, the gains expected by hybridization with vision are:
Attitude: error reduction of 35% Position: error reduction of 90% Speed: error reduction of 85% Strong gains are expected in z (better than GPS)
Average standard error in z: 30 cm! (between 5cm and 2m on a typical scenario)
26 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
OBSTACLE WARNING SYSTEMARCHITECTURE À L’ÉTUDE
AOWSEV_SV Fusion
Symb Processor
DEUs
Terrain Database Generator
and SV ProcessorFlight Control
Computers
ACFT Geo PositionMAGRS/EGI (GPS)
I2HMD (COTS/GDE)
DTED 2+ MSD
DIRECT
HMD Tracked FOR - Display
I2HMDProc
HMD Head Tracker
DTED/HRTI Update
Synthetic Vision Data
DigMap Dig
Map
I2HMDProc
HMD Head Tracker
POWS
1553/Ethernet
1553/EthernetCPLD
G RO WTH
GR
OW
TH
A U T O N A C
HVR CPLD REM HO VER
O N
A R M
ACT
CPLD
I NC
DECR
VS LM TCRS HDG
HO LD
SPEED
HO LD
ALT
HO LD
SRCH PAT WYPTAPPR
E
N
A
V
I
N
A
V
TACANVO R/ I LSG RO WTH
GR
OW
TH G R
BRTA R M
ACT
A R M
ACT
A R M
ACT
A R M
ACT
A R M
ACT
A R M
ACT
O NO NO N
P U S H
X 1 0
O NO N
CO
NT
RO
L P
AN
EL
AR
EA
PE
R
PH
OT
O E
TC
H
DR
AW
ING
S E L
T
F A IL
O F F
O N
B R T
12XXX
USABLE FUELFEED
XXXXX
FLAPSENG
TRQ
NR
XXX
NP
XXX
XXX
90
HYD 2
FAI LHYD 3
FAI L
HYD 1
FAI L
EST EST
XXX
NPNP
XXX
FEED
EST
XX XX
A LT
01
3
45
6
7
8
9
9
99 600
6 08 0
1 0 0
1 2 0
1 4 0
1 6 0
2 0 0
1 8 02 5 0
3 0 0
K N O T S
IA S
3 5 0
Y
2 3
5 6
8 9
A B
H
O P
C D
J K
Q R
W X
E F
L M
S T
Z
1
4
G
N
U
7
V
/ 0
IN A V E N A V
F P L N WY P T E C S
IF F M S N W/W
R T N
E M C O N
C O M M
D E L C L R
T IM E R
E M E R
L T S
E N T E R
ID E N T
P G
P G
S E Q
S E Q
Y
2 3
5 6
8 9
A B
H I
OP
C D
J K
Q R
W X
E F
L M
ST
Z
1
4
G
N
U
7
V
/ 0
IN A V E N A V
F P L N WY P T E C S
IF F M S N W/W
R T N
E M C O N
C O M M
D E L C L R
T IM E R
E M E R
L T S
E N T E R
ID E N T
P G
P G
S E Q
S E Q
C HC H
BR
T
C HAN RFILC R STB Y SECACT I VES
E
L
C HC H
BR
T
C HAN RFILC R STB Y SECACT IVES
E
L
C D U A C K C D US Y S
S T A TC /AE IC A S
C A C
B R T
O F F
N IG H T
D A Y
C A C
CO M M 1
01N/ 251. 000
CO M M 1
13 N/ 123. 50
VO R/ I LS FREQ
110. 30
TCN CHAN
123X
BARO PRES
29. 92RAD ALT
LO W SET
200
M O DE 3 CO DE
7600C
X X X X
M G T
X X X X
N G
X X XX X X X X X
X X X
X X X
X X X XX X X XX X X X
XXX XXX
X X X
P S I
X X X
O IL
P S I
P R G B
T A G B
P S I
C
H Y D 1 H Y D 2 H Y D 3
P S I
C
C
XXX
XXX XXX
XXX
XXXO
O
O
APU O N
EM CO N
RAM P O PEN
M WG B
DC BUS 1 FAI L
AFT CRO WN O PEN
HY D SYS 1 PRESS HI
L FEED TANK LO
RADAR BEACO N FAI L
HY D SYS 1 PRESS HI
HY D SYS 1 PRESS HI
CO M M 1
01N/ 251. 000
CO M M 1
13 N/ 123. 50
VO R/ I LS FREQ
110. 30
TCN CHAN
123X
BARO PRES
29. 92
RAD ALT
LO W SET
200M O DE 3 CO DE
7600C
PFD NAV FLI R STAT
AVI O NI CS PO WER SELECT
SYST*
AVI O NI CS EQ UI PM ENT PO WER
PO WER
O FF
PWR
ALL
O N
SEQ
O FF
O N
O N
O N
O N
O N
O N
O N
O N
O N
O N
O N
ADF
ARC 210 #2
DEU - CO PI LO T
DM S
FM HO M I NG
I FF
I NS 1
I NS 2
I NS 3
RAD ALT
SATCO M
TACAN
S Y M
C O N B R T
A C KC /A
O F F
N IG H T
D A Y
S Y M
C O N B R T
A C KC /A
O F F
N IG H T
D A Y
S Y M
C O N B R T
A C KC /A
O F F
N IG H T
D A Y
S Y M
C O N B R T
A C KC /A
O F F
N IG H T
D A Y
S L E W
P R E S S D E S IG N A T E
S L O W
F A S TN F O V W
P O L
HDD HDDDisplay System (CMS)
1553/Ethernet
DIRECT
HMD Tracked FOR - Display
DEUs
Current EquipmentAugmenting Equipment
Optional Equipment
DEUs : Display Electronic UnitsDTED : Digital Terrain Elevation DataEV : Enhanced VisionSV : Synthetic VisionHMD : Helmet Mounted DisplayCMS : Cockpit Management SystemDAS : Distributed Aperture SystemHRTI : High Resolution Terrain InformationPOWS : Passive Obstacle warning SystemAOWS : Active Obstacle Warning SystemHDD : Head Down Display
DVE SYSTEM ARCHITECTURE (NIAG SG 167)
DAS Uncooled-IR / LLLTV
27 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Conclusion
A Passive Optronic System to increase Situation Awareness
to strengthen the traffic separation capability using non-cooperative and independent means
OK, by using distributed staring sensors and air surveillance algorithms
to offer a credible, compact and low cost alternative compared to current obstacle warning systems based on LASER or RADAR
OK, by using distributed staring sensors and SLAM/Stereovision algorithms, the system can provide alerts when detecting obstacles on ground
to provide solutions to extend HTAWS and SVS operational credit at low altitude.
OK, by using SLAM / stereovision Algorithms : the system may elaborate a “user” 3D mapping of the neighborhood and may improve the accuracies of the navigation data (INS/Vision hybridization, Terrain Aided Navigation)
to provide day / night enhanced vision for HDD, HuD, Helmet Mounted Display…
28 /
Ce document et les informations qu’il contient sont la propriété de Sagem. Ils ne doivent pas être copiés ni communiqués à un tiers sans l’autorisation préalable et écrite de Sagem.
Questions ?