ConcourseCI love Minio

30/11/2016 Etourneau Gwenn

@The_shinji62

ConcourseCI

Minio

About meEtourneau Gwenn

Sr Solution Architect

Bef.: Platform Architect

https://github.com/shinji62

https://twitter.com/the_shinji62

Pivotal

• Spring Framework • RabbitMQ • Concourse

• Cloud Foundry • Pivotal Tracker • Gemfire

Meetup

Agenda

• Concourse • Minio • Minio for Concourse • Q&A

ConcourseCI

ConcourseCI

• Do not allow snowflake • Isolation between job • Non share volume / storage / cache

Jobs 1 Jobs 2

file_out file_outX

Concourse

Concourse

• Concourse support S3 • S3 on premise is not easy • RiackCS, leoFS, Swift are hard • S3 compatibility so so…

Minio

Minio• Open Source • Object Storage written in Go • Distributed ? Not really more like replication • S3 compatible• Data protection using Bit rots , Erasure Code • Single go binary easy to start

Minio limitation

• Do not scale-out or scale back • Not multi-tenant, user are the same for the same pool • number of nodes =>4 and <= 16 • Still early phase (>5 commit /day on master)

Minio 1 disk by node

Minio Many disk by nodes

Minio and Concourse• Support S3 native resource from concourse

Jobs 1Jobs 2

file_outfile_out

file_out

Minio bosh release

Minio bosh release• Deploy a pool of distributed minio in minutes • On any IaaS (supported by bosh)

Minio bosh release• Use bosh v2 links • Easy to spin-off • well, you need to learn bosh a little….

Minio 1 disk by node

Demo

Demo

• Deploy Minio Cluster

• Set Concourse Pipeline

Resources

https://github.com/shinji62/minio-dist-boshreleasehttps://github.com/cloudfoundry-community/minio-boshrelease

• Minio bosh release

• Minio • https://github.com/minio/minio • https://minio.io/

Resources

• ConcourseCI • https://concourse.ci

Slack

https://concourseci.slack.com/

#general

Q & A

Thank You !!