Loading…
This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
View analytic
Thursday, May 15 • 5:00pm - 5:40pm
Future of EDP: plugins, SPI, Oozie

Sign up or log in to save this to your schedule and see who's attending!

1) Plugins SPI review

Sahara consists of different layers:

1. cluster provisioning
2. EDP

Most probably new layers will appear:

3. hadoop security management
4. non-hadoop activities

How plugin SPI should look to support all kinds of communication? Should it be one SPI or several SPIs? Do we allow communication by naming convention (e.g. expect something with given name in extra field)?

===========================

2) EDP plugins - get rid of Oozie hard dependency

We should make EDP pluggable to be able to use not only Oozie in EDP. It's needed to support running Spark / Storm workloads in future using EDP on top of Sahara's provisioned clusters.

===========================

3) Future of EDP

Now we assume that:
1. oozie is used for workflow management
2. all plugins potentially support all job types (may be not implemented)

Do we continue with such assumptions?
What other job types need to be added in Juno?

(Session proposed by Andrew Lazarev)


Thursday May 15, 2014 5:00pm - 5:40pm
B304

Attendees (12)

  • Profile image