Cloud State

Cloud State

Purpose and high level description of Cloud State is pretty well described in attached Angus’ mail, so pasting it here:

The idea is to encapsulate the actual launching, terminating and status tracking of instances in a discrete rails engine. By separating it from the main code, it can form a standalone cloud state service. This is a similar approach to the work which Martyn is currently doing, defining an image management engine. He’s moving a lot of existing functionality into rails engine.

The benefits of this approach would be:

• We’d have a stateful API to cloud providers, addressing the constraints that the fact that deltacloud isn’t stateful has presented. The cloud state service could be configured to send callbacks to Conductor when there are changes in an instance’s state.

• By defining the interface to the cloud state service, and articulating the API by which Conductor, and whatever other apps might need to in future, interact, we have achieve a separation which will be valuable in future. For instance, when the Heat project is completed, we could use the same API to interact with Heat from Conductor.

• it could evolve into a more separate service, which is addressable by imagefactory etc. and proxies all the traffic to the cloud.

The sequence of events would be:

After Conductor has carried out HWP matching, provider account selection, quote and permissions checking, verified the availability of images etc., Conductor would sends a launch order to the cloud-state engine with UUIDs of things to launch and user-data to pass to each instance, and asks the state engine to notify it of state-change events for each instance it launches. The launch order would include cloud account credentials for each cloud provider account the state engine would launch on (there might be more than one eventually).

As each instance is launched, the state engine pokes Conductor with status updates, and its IP address and ssh key.

During the lifecycle of the app, the cloud state service monitors all running instances and updates listeners with state changes.

We’d need to make some sensible decisions around how long the cloud state service would cache provider account credentials for, if at all, as well as specifying the API between Conductor and the cloud state service. It is worth bearing in mind that we might want to allow the API to support addressing multiple instances in a single call.

In the first instance, I’d suggest that this doesn’t need to be more than a thin API on top of delayed-job. Creating it in a rails engine would allow for the separation which designing an API that works requires without creating too much additional work.

Implementation notes

Following text describes some high level implementation notes. This scenario is related to other existing scenarios:

• Dbomatic_testability_and_performance
• Robust_instance_launching

Cloud state service in fact wraps/isolates functionality which is now included in app/util/taskomatic and dbomatic service. Rails::Engine might be used for better isolation of this functionality and it will allow us to run this as a standalone service in future. In first iteration we might implement only model part of Rails::Engine. Because this engine will be included in Conductor, we can call model methods of Cloud State directly. In some next iteration we add REST API (probably CRUD controllers actions).

Tasks

• create gem+rpm for cloud state rails engine
• create basic cloud state models
• move app/util/taskomatic logic into cloud state models, update references in Conductor’s code to reflect this change
• move dbomatic logic to cloud state

Models

We need 2 models: CloudStateInstance, CloudStateCallback (TODO: use better names, but distingush these names from common Condustor model names). Both models inherit from Activerecord::Base, so common methods (find, create, destroy, ...) can be used.

• CloudStateInstance - represents instances which should be checked for changes

  • Has many callbacks, dependent => destroy

  • Attributes (subset of DC API instance attributes which we check for changes):

    • id
    • uuid (uuid is generated and passed from Conductor so we can pair this instance with Conductor's instance)
    • state
    • public_address
    • private_address
    • external_key
    • TODO: for first iteration, we don't keep account credentials, for launch/check we fetch this info directly from Codnuctor's model
    • this method creates CloudStateInstance object and also sends create_instance request (=launch) to a provider

CloudStateInstance.launch({ :uuid => uuid, :image_id => DC-API image ID, :name => instance name, :hwp_id => DC-API hwp ID, :hwp_memory => overridesdefault memory(optional), :hwp_cpu => overrides default cpu (optional), :hwp_storage => overrides default storage (optional), :keyname => ssh key name id (optional), :callback => method name (optional) (TODO: will be URI once standalone service is supported), if present callback is automatically created })

• CloudStateCallback - when instance is changed, all instance's callbacks are triggered

  • Belongs to CloudStateInstance

  • Attributes

    • id
    • instance_id
    • action (TODO: method name to be called? -> restrict to some subset because of security?)

• CloudStateInstance#do_action(action) (instance method)

  • actions: start|reboot|stop

  • validates if the action is valid, then sends this action to DC-API

  • raise an exception if the action is invalid

  • TODO: sync vs async? if this method is async, what if an exception occurs (eg provider is unreachable)?

    CloudStateCallback.register({ :instance_id => CloudStateInstance instance object id (optional) :instance_uuid => instance's uuid (optional, if instance_id is not set, instance is found by uuid), :action => method name (this method will be called when the instance is changed) (TODO: will be replaced with URI in future maybe), })

  • Cloud state checks state for all instances which are in “cloudstateinstances” table (TODO: check only not-stopped instances?).

(next iteration) Add controllers actions which enables Cloud State to be standalone service with RESTful API ~~> 2 controllers for instances and callbacks, each of them with CRUD actions

TODO:

• use cloud state for checking Realms availability?