Juno Design Summit has ended
Back To Schedule
Thursday, May 15 • 5:00pm - 5:40pm
Future of EDP: plugins, SPI, Oozie

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

1) Plugins SPI review

Sahara consists of different layers:

1. cluster provisioning
2. EDP

Most probably new layers will appear:

3. hadoop security management
4. non-hadoop activities

How plugin SPI should look to support all kinds of communication? Should it be one SPI or several SPIs? Do we allow communication by naming convention (e.g. expect something with given name in extra field)?


2) EDP plugins - get rid of Oozie hard dependency

We should make EDP pluggable to be able to use not only Oozie in EDP. It's needed to support running Spark / Storm workloads in future using EDP on top of Sahara's provisioned clusters.


3) Future of EDP

Now we assume that:
1. oozie is used for workflow management
2. all plugins potentially support all job types (may be not implemented)

Do we continue with such assumptions?
What other job types need to be added in Juno?

(Session proposed by Andrew Lazarev)

Thursday May 15, 2014 5:00pm - 5:40pm EDT

Attendees (0)