Morpheus Plugin Documentation

The Morpheus Plugin API is a Java 8 based library for creating Plugins that add functionality to Morpheus. The Plugin api supports implementing providers to Morpheus of the following types:

Developing plugins for Morpheus is a bit different than simply creating custom tasks or custom library items in Morpheus. It requires some programming experience. We should preface with the fact that Morpheus runs on top of Java. It is primarily written in a dynamic language called Groovy and is based on Groovy version 3.0.7 currently running within Java 11 (openjdk). The provided plugin api dependency provides a common set of interfaces a developer can implement to dynamically load functionality into the product. These developers are free to develop plugins in native Java or leverage the groovy runtime. Most examples provided are developed and sampled in groovy. It is a great dynamic language that is much less verbose and easier to understand than native java based languages. Plans for supporting additional languages are in the works. Most notably kotlin is being looked into as an alternative development platform. jRuby is also a viable language that can be used as the runtime is included with Morpheus. The plugin architecture is designed based on reactive models using rxJava2. This library follows ReactiveX models for Observer pattern based programming. One unique thing about Observer pattern programming is to remember when writing a call to an Observable the code is not executed until the final subscribe call. If this call is missing, the code will never execute.

NOTE: rxJava3 is a different project build and cannot be used yet in place of rxJava2.

The plugin pipeline leverages Gradle as the build tooling. Gradle is a cross-platform programmatic build tool that is very commonly used and is most notably also used in the android space. To begin make sure your development environment has Gradle 6.5 (at least but if using newer Gradle make sure you are familiar with the definition changes needed) as well as Java 11 (if using openjdk over 11 make sure target compatibility is set to 1.11 within your project).

NOTE: Currently Gradle 7.x is recommended as we upgrade support to Gradle 8

The morpheus-plugin-core git repository is structured such that sample plugins exist underneath the samples directory. They are part of a multi-project gradle project as can be defined in the root directories settings.gradle. This adds a bit of complexity and should be ignored when developing ones own plugin. Simply start with a blank project folder and a simple build.gradle as demonstrated below.

The structure of a Plugin is a typical "fat jar" (aka shadowJar). The plugin will include morpheus-plugin-api along with all the dependencies required to run. Though not required, it is often conventional to use the groovy programming language when developing plugins as can be seen in the samples. Morpheus is based on the groovy runtime and therefore allows full use of groovy 3.0.x.

The Structure of a project often starts with a gradle build along with a Plugin class implementation and a manifest that points to this class. This is the entrypoint for the plugin where all metadata about the plugin is defined as well as all Providers offered by the plugin are registered.

The Morpheus context allows you to interact with, query and save data with Morpheus. It is organized into several sub service classes to perform operations that may involve database calls or calling common methods within the Morpheus core that are not available directly. Some calls may be as simple as listById or save. But they can also be as complex as executeSshCommand or executeWinrmCommand. All calls within a Morpheus Context implement RxJava2 conventions. For details on how to program in Reactive syntax and RxJava concepts please see the documentation site http://reactivex.io/.

When interacting with various subcontexts it is helpful to know there is a common guideline on method names involving database calls. These common method names include:

There are, of course, exceptions and non ORM related methods that also may exist in certain services that provide common helper methods. For a full listing of the various service classes please check the API Docs and look at the MorpheusContext class. There are several in line with general ORM calls as well as some related to certain actions.

When interacting with Morpheus you must use the models provided in com.morpheusdata.model.*. This allows for the context to be strongly typed. These Models have a few conventions that you might like to know about. Firstly, models that are often synced from a cloud provider or integration of any kind often inherit from their base class known as an IdentityProjection which also inherits from MorpheusModel.

For Example: the NetworkDomain model inherits NetworkDomainIdentityProjection which in turn inherits MorpheusModel.

The IdentityProjection contains a subset of properties that are typically used to match the object with its equivalent on the other side of an Api implementation. This is typically persisted in the externalId property of the object. Some objects also leverage uniqueId as well.

It is also recommended to use the getter and setter methods on the Models (which should be strictly required in the near future) so as to ensure the dirtyProperties map is updated appropriately. This will be used in future distributed worker installations to reduce bandwidth in transmission of data updates back to Morpheus.

As the Morpheus Plugin API has progressed a new concept was created in 0.15.x. It was determined a consistent means for querying Morpheus data objects needed to be made across the board in all of the MorpheusContext accessible services. Previously, there were only a fixed set of options per Model that one might have wanted to query. But with the development of the DataService model and the new DataQuery object, a lot more power is available to the developer.

These services are now commonly used in various sync related activities with SyncTask as well as can be used with custom reporting or the new DataSetProvider concepts.

Above you will see a common set of methods for querying data. The important part of these methods is to note that they all take a DataQuery object. This object allows you to scope a query to a User or an Account within morpheus as well as pass in complex query operators.

For Example:

The above query looks for all clouds that contain a set of usage account ids via either the externalId property or the linkedAccountId property.

As can be seen above, queries can be nested with DataOrFilter or DataAndFilter combinations to build complex queries. More details of this also exist on the DataQuery java doc.

NOTE: The DataService also provides save, create,remove methods of a consistent nature. Please refer to the APIDoc / javadoc for descriptions on how to use these methods.

In the past, when making custom reports, it was common to use the direct database connection and query the morpheus data set directly. This carried with it some risks as these tables are not strictly documented and some data is in an encrypted state. By utilizing data services ( where possible ), however, the object models are marshalled into a properly documented format and fields that may have been previously inaccessible, due to encryption, are now available for use.

Many of the Morpheus Provider types have a method to periodically refresh and sync data into Morpheus. This can be useful for representing core concepts or doing brownfield discovery for things like ComputeServer or NetworkPool objects. Also, it may be useful for syncing in user presented options in a dropdown or typeahead during a particular operation (Generic Data for this we can store n ReferenceData).

Over the course of several years, Morpheus has developed an optimal way to efficiently sync data from a remote endpoint into the appliance. This has to take into account network bandwidth, memory , cpu, and load on the target api. Morpheus also created a helper class for making sync operations simple and consistent called the SyncTask (javadoc available).

Below is an example that syncs available amazon regions in the AWS Cloud to the CloudRegion table.

NOTE: The above example does not show that the api query to get all regions was performed above.

A SyncTask is capable of taking the Identity objects of a class (small form of a Model that contains just the important identification fields used for matching) and comparing them against the api results list. This is done via the addMatchFunction as seen above.

NOTE: It is possible to have a chain of match functions as secondary fallbacks by simply adding another to the chain.

Objects that do not have a match from the api are sent in batches to the onAdd for creation of Morpheus Model objects and consequently the inverse is true for models that are not found within the api results as these are sent to onDelete.

Objects that do match are a bit different. Firstly, in order, to check the object for changes it must first be fully loaded from its Identity. This can be seen in the withLoadObjectDetailsFromFinder call. This translates those Identity classes into their fully loaded objects in batches of 50 at a time (not seen as this is magically done by the SyncTask). Once these objects are fully loaded they are passed down into the onUpdate where they can be compared.

Although not seen above, it is best to bulk create or bulk save morpheus changes in these chunked add and update methods. This is done using the MorpheusDataService.bulkCreate or MorpheusDataService.bulkSave implementations on the context.

Sync methods such as this exist in several provider types beyond just clouds. IPAM Providers, DNS Providers, backups and others all contain endpoints where these sync operations can be performed.

A plugin may register an endpoint or endpoints to respond with html or json. To register your routes in your plugin you must implement the PluginController class.

The morpheus-plugin-api library comes with a utility for facilitating quick and easy API calls to external integrations. This is called teh HttpApiClient and replaces the use of the RestApiUtil.

The developer is, of course, able to utilize any HTTP or SDK/API client they choose within the java ecosystem if so desired. However, this client provides some common functions such as SSL Validation, Throttling, Keep-Alive, Etc. and is based on the Apache HTTP Client.

Plugins may render html sections such as adding a tab to different areas of Morpheus which you can populate with your own content. By default a Handlebars template provider is provided by the Plugin Manager. If you wish to use your own template engine you may implement com.morpheusdata.views.Renderer interface.

Morpheus Plugins support using i18n localization properties. These are string maps for representing various sections of text in your plugin (or in morpheus itself) by a localized way. Creating a messages.properties file in the directory src/main/resources/i18n will allow you to set some key value maps.

For Example messages.properties:

or for Spanish (ES) messages_es.properties:

These can be leveraged in the handlebarsRenderer for server side rendering via the {{i18n}} helper like so:

Or they can also be used in javascript rendering (like in dashboard widgets or other areas):

If a matching string exists for the current browser locale, it will be used, if not it will fall back to the default messages.properties file.

A DataSetProvider is a server side call to load a dynamic data for custom form inputs, as well as data references with scribe templates. A dataset can feed data to dropdown lists, multiselect components, and typeahead components. Datasets can be very useful when designing task types that have custom options or even custom catalog item layouts. In the past, this type of data was provided with an OptionSourceProvider which contained multiple definitions in a single provider, but had its own limitations and did not provide sufficient documentation on the data provided. The DataSetProvider improved dynamic data definitions by: isolating one dataset per provider, including associative information so that it can be used for export/import scribe functionality, and including documentation on the dataset within the dataset definition.

NOTE These have been replaced by DataSet Providers as of 0.15.x.

DEPRECATED: An Option source is a server side call to load a dynamic dataset for custom form inputs. These normally feed dropdown lists, multiselect components, and typeahead components. These can be very useful when designing task types that have custom options or even custom catalog item layouts.

During the 5.4.x and 5.5.x release of Morpheus. Credentials were introduced. It became possible to store integration credentials externally or decoupled from a specific integration. This is great for service accounts! As a result a new option type was created to allow them to be used for custom plugin integrations.

We recommend Spock for easily testing your plugin. The interfaces can easily be mocked and stubbed to allow you to test your integrations without a running Morpheus instance.

If you need to ensure that certain data (e.g. layouts, plans, etc.) are added to the database when your plugin is installed, you can use this process to "seed" that data. Morpheus consumes and produces HCL formated data. The same principles that you can use here for plugins also apply to Morpheus packages.

For plugins, you will need to create a folder called scribe in your your plugin’s resources folder (src/main/resources/scribe). Then add HCL seed files to that folder. We recommend descriptive names like Ubuntu22Layouts.scribe. The scribe files should be modeled off of plugin database model classes. The resource type should be the model name in dash-case and the resource should have a unique identifier (the code field for most models). All new scribe files and updates to existing those files will be processed when your plugin is installed or reloaded.

Resources can reference other resources in the same .scribe file, other resources from different .scribe files, resources from other plugins, and even existing resources in Morpheus. The references are made by specifying the resource type and a unique identifier, often the code. Just make sure that if your plugin expects resources from other plugins or packages, those plugins or packages are loaded first.