Une infrastructure de stockage et sa suite analytique : Le duo gagnant du Datalake Foundation

© Copyright 2015 EMC Corporation. All rights reserved © Copyright 2015 EMC Corporation. All rights reserved

JEUDI 19 NOVEMBRE 2015 Denis FRAVAL-OLIVIER : ISD Presales Manager

DATA LAKE FOUNDATION 2.0


Module 4: Horizontal and Vertical Markets

EMC Isilon – Unifying Workloads in one place

© Copyright 2015 EMC Corporation. All rights reserved

Archive & Backup Target

File shares Home Directories

BLOBS

Design, Test & Manufacture

Consumerization Personalization

Splunk

Processes & Transaction

Hadoop & Analytics

Sync ‘n Share

Demographics

Web Content

Social & Next-Gen

Surveillance

ISILON FOR ALL TYPES OF DATA

http://4.bp.blogspot.com/-AJCDbLLOOSs/TVro7TAqvrI/AAAAAAAAAGI/bE0v-UIA9Fg/s1600/sharepoint+2010.png


EMC Isilon Scale-Out platform

Clients and Applications

RESTful API GET PUT POST DELETE

Gig-e 10 Gig-e Network

OneFS Operating Environment

Multi-Protocol Client/Application Layer Ethernet Layer

Protocols

SMB NFS

FTP HTTP

HDFS for Hadoop

REST for Object

Intra-cluster Communication

© Copyright 2015 EMC Corporation. All rights reserved

DATA LAKE EMC ISILON SCALE-OUT NAS

DATA PROTECTION

DATA SECURITY PERFORMANCE MANAGEMENT

DATA MANAGEMENT

Data Lake

S-Series X-Series

NL-Series HD-Series

5


S - Series X - Series

NL-Series

Isilon CloudPools

3rd Platform cloud Innovation

HD-Series

6

FUTURE


NFS

HDFS

SMB, NFS, HTTP, FTP, HDFS

Node reply Node reply Node reply Node reply

Support for Multiple Analytics Applications

name node

name node

name node

name node

data n

od

e

NFS

SMB

SMB

NFS MAP Reduce

MAP Reduce

MAP Reduce

MAP Reduce

MAP Reduce

MAP Reduce


Splunk Index Architecture


The Big, Cold Data Lake


Benefit: Unmatched Scalability = Unmatched Simplicity

Cold Isilon

Single Volume scaling to

50PB

****.gz

****.tsidx

The “Bottomless” Cold Bucket


Use Splunk Multi-Site Clustering For Site Protection

Use Array Snapshots For Data Protection

****.gz

****.tsidx

Snapshots

Benefit: Snapshots + Splunk Replication = NO BACKUPS

Hot/Warm XtremIO

****.gz

****.tsidx

Splunk Multi-Site

Clustering

Backupless “Bliss”

Cold Isilon


Using Self Encrypting Drives (SED) AES256 Encryption Algorithm

Drives Taken Out Are Unreadable

Benefit: Encryption = Piece Of Mind

Hot/Warm XtremIO

Cold Isilon

Bonus !! Encryption


Benefit: Automation = Simplicity

Automated Tiering For High Density Capacity Keep Data In Cold Then Delete

Write-Once-Read-Many (WORM) protection SEC Rule 17a-4(f) definition standards

Bonus !! Always Searchable

Cold Isilon


Data-in-place analytics Multi-protocol access

Efficiency gains of 20% vs 300% overhead Enterprise Features For Hadoop

Benefit: Isilon HDFS = SIMPLE Hadoop Analytics

Bonus !! HUNK Ready…


Hadoop Market Leadership

Market Leader in Hadoop Shared Storage #1

Customers 700+

YoY Growth 250%


Hbase Storm Hive Pig Map Red

YARN

Tez Sqoop Knox Spark Kafka

NameNode

HADOOP ARCHITECTURE - TRADITIONAL

Ethernet

Data Node + Compute Node







Ethernet

DataNode

Compute Node Compute Node Compute Node

Compute Node Compute Node Compute Node

NameNode

name node

name node

name node

name node d

ata no

de

Hbase Storm Hive Pig Map Red

YARN

Tez Sqoop Knox Spark Kafka

Ambari Agent

HADOOP ARCHITECTURE – WITH ISILON


Traditional Hadoop - Layers


Isilon + Hadoop – NO Layers




HDFS

SMB, NFS, HTTP, FTP, HDFS

Node reply Node reply Node reply Node reply

HDFS: Integrated Isilon and vHadoop

name node

name node

name node

name node

data n

od

e

NFS

SMB

SMB

NFS

Apache


Si vous avez des questions sur cette présentation, n’hésitez pas à prendre contact directement contact

avec :

Denis FRAVAL-OLIVIER [email protected]

Technology

Une infrastructure de stockage et sa suite analytique : Le duo gagnant du Datalake Foundation