NAME

    Myriad - microservice coördination

SYNOPSIS

     use Myriad;
     Myriad->new->run;

DESCRIPTION

    Myriad provides a framework for dealing with asynchronous,
    microservice-based code. It is intended for use in an environment such
    as Kubernetes to support horizontal scaling for larger systems.

    Overall this framework encourages - but does not enforce -
    single-responsibility in each microservice: each service should
    integrate with at most one external system, and integration should be
    kept in separate services from business logic or aggregation. This is
    at odds with common microservice frameworks, so perhaps it would be
    more accurate to say that this framework is aimed at developing
    "nanoservices" instead.

 Do you need this?

    If you expect to be dealing with more traffic than a single server can
    handle, or you have a development team larger than 30-50 or so, this
    might be of interest.

    For a smaller system with a handful of users, it's probably overkill!

Modules and code layout

      * Myriad::Service - load this in your own code to turn it into a
      microservice

      * Myriad::RPC - the RPC abstraction layer, in $self->rpc

      * Myriad::Storage - abstraction layer for storage, available as
      $self->storage within services

      * Myriad::Subscription - the subscription handling layer, in
      $self->subscription

    Each of the three abstractions has various implementations. You'd set
    one on startup and that would provide functionality through the
    top-level abstraction layer. Service code generally shouldn't need to
    care which implementation is applied. There may however be cases where
    transactional behaviour differs between implementations, so there is
    some basic functionality planned for checking whether
    RPC/storage/subscription use the same underlying mechanism for
    transactional safety.

 Storage

    The Myriad::Storage abstract API is a good starting point here.

    For storage implementations, we have:

      * Myriad::Storage::Redis

      * Myriad::Storage::PostgreSQL

      * Myriad::Storage::Memory

    Additional transport mechanisms may be available, see CPAN for details.

 RPC

    Simple request/response patterns are handled with the Myriad::RPC layer
    ("remote procedure call").

    Details on the request are in Myriad::RPC::Request and the response to
    be sent back is in Myriad::RPC::Response.

      * Myriad::RPC::Redis

      * Myriad::RPC::PostgreSQL

      * Myriad::RPC::Memory

    Additional transport mechanisms may be available, see CPAN for details.

 Subscriptions

    The Myriad::Subscription abstraction layer defines the available API
    here.

    Subscription implementations include:

      * Myriad::Subscription::Redis

      * Myriad::Subscription::PostgreSQL

      * Myriad::Subscription::Memory

    Additional transport mechanisms may be available, see CPAN for details.

 Transports

    Note that some layers don't have implementations for all transports -
    MQ for example does not really provide a concept of "storage".

    Each of these implementations is supposed to separate out the logic
    from the actual transport calls, so there's a separate ::Transport set
    of classes here:

      * Myriad::Transport::Redis

      * Myriad::Transport::PostgreSQL

      * Myriad::Transport::Memory

    which deal with the lower-level interaction with the protocol,
    connection management and so on. More details on that can be found in
    Myriad::Transport - but it's typically only useful for people working
    on the Myriad implementation itself.

 Other classes

    Documentation for these classes may also be of use:

      * Myriad::Exception - generic errors, provides "throw" in
      Myriad::Exception and we recommend that all service errors implement
      this rôle

      * Myriad::Plugin - adds specific functionality to services

      * Myriad::Bootstrap - startup used in myriad.pl for providing
      autorestart and other functionality

      * Myriad::Service - base class for a service

      * Myriad::Registry - support for registering services and methods
      within the current process

      * Myriad::Config - general config support, commandline/file/storage

METHODS

 loop

    Returns the main IO::Async::Loop instance for this process.

 services

    Hashref of services that have been added to this instance, as name =>
    Myriad::Service pairs.

 configure_from_argv

    Applies configuration from commandline parameters.

    Expects a list of parameters and applies the following logic for each
    one:

      * if it contains :: and a wildcard *, it's treated as a service
      module base name, and all modules under that immediate namespace will
      be loaded

      * if it contains ::, it's treated as a comma-separated list of
      service module names to load

      * a - prefix is a standard getopt parameter

 redis

    The Net::Async::Redis (or compatible) instance used for service
    coördination.

 memory_transport

    The Myriad::Transport::Memory instance.

 rpc

    The Myriad::RPC instance to serve RPC requests.

 rpc_client

    The Myriad::RPC::Client instance to request other services RPC.

 http

    The Net::Async::HTTP::Server (or compatible) instance used for health
    checks and metrics.

 subscription

    The Myriad::Subscription instance to manage events.

 storage

    The Myriad::Storage instance to manage data.

 registry

    Returns the common Myriad::Registry representing the current service
    state.

 add_service

    Instantiates and adds a new service to the "loop".

    Returns the service instance.

 service_by_name

    Looks up the given service, returning the instance if it exists.

    Will throw an exception if the service cannot be found.

 ryu

    a source to corresponde to any high level events.

 shutdown

    Requests shutdown.

 on_start

    Registers a coderef to be called during startup. The coderef is
    expected to return a Future.

 on_shutdown

    Registers a coderef to be called during shutdown.

    The coderef is expected to return a Future indicating completion.

 run_future

    Returns a copy of the run Future.

    This would resolve once the process is running and it's ready to accept
    requests.

 shutdown_future

    Returns a copy of the shutdown Future.

    This would resolve once the process is about to shut down, triggered by
    a fault or a Unix signal.

 setup_logging

    Prepare for logging.

 setup_tracing

    Prepare OpenTracing collection.

 run

    Starts the main loop.

    Applies signal handlers for TERM and QUIT, then starts the loop.

SEE ALSO

    Microservices are hardly a new concept, and there's a lot of prior art
    out there.

    Key features that we attempt to provide:

      * reliable handling - requests and actions should be reliable by
      default

      * atomic storage - being able to record something in storage as part
      of the same transaction as acknowledging a message

      * flexible backends - support for various storage, RPC and
      subscription implementations, allowing for mix+match

      * zero transport option - for testing and smaller deployments, you
      might want to run everything in a single process

      * language-agnostic - implementations should be possible in languages
      other than Perl

      * first-class Kubernetes support - k8s is not required, but when
      available we should play to its strengths

      * minimal boilerplate - with an emphasis on rapid prototyping

    These points tend to be incompatible with typical HTTP-based
    microservices frameworks, although this is offered as one of the
    transport mechanisms (with some limitations).

 Perl

    Here are a list of the Perl microservice implementations that we're
    aware of:

      * https://github.com/jmico/beekeeper - MQ-based (via STOMP), using
      AnyEvent

      * https://mojolicious.org - more of a web framework, but a popular
      one

      * Async::Microservice - AnyEvent-based, using HTTP as a protocol,
      currently a minimal wrapper intended to be used with OpenAPI services

 Java

    Although this is the textbook "enterprise-scale platform", Java
    naturally fits a microservice theme.

      * Spring Boot <https://spring.io/guides/gs/spring-boot/> - One of the
      frameworks that integrates well with the traditional Java ecosystem,
      depends on HTTP as a transport. Although there is no unified storage
      layer, database access is available through connectors.

      * Micronaut <https://micronaut.io/> - This framework has many
      integrations with industry-standard solutions - SQL, MongoDB, Kafka,
      Redis, gRPC - and they have integration guides for cloud-native
      solutions such as AWS or GCP.

      * DropWizard <https://www.dropwizard.io/en/stable/> - A minimal
      framework that provides a RESTful interface and storage layer using
      Hibernate.

      * Helidon <https://helidon.io/> - Oracle's open source attempt,
      provides support for two types of transport and SQL access layer
      using standard Java's packages, built with cloud-native deployment in
      mind.

 Python

    Most of Python's frameworks provide tools to facilitate building logic
    blocks behind APIs (Flask, Django ..etc).

    For work distribution, Celery
    <https://docs.celeryproject.org/en/stable/> is commonly used as a task
    queue abstraction.

 Rust

      * https://rocket.rs/ - although this is a web framework, rather than
      a complete microservice system, it's reasonably popular for the
      request/response part of the equation

      * https://actix.rs/ - another web framework, this time with a focus
      on the actor pattern

 JS

    JS has many frameworks that help to implement the microservice
    architecture, some are:

      * Moleculer <https://moleculer.services/> - generally a
      full-featured, well-designed microservices framework, highly
      recommended

      * Seneca <https://senecajs.org/>

 PHP

      * Swoft <http://en.swoft.org/> - async support via Swoole's
      coroutines, HTTP/websockets based with additional support for
      Redis/database connection pooling and ORM

 Cloud providers

    Microservice support at the provider level:

      * AWS Lambda <https://aws.amazon.com/lambda> - trigger small
      containers based on logic, typically combined with other AWS services
      for data storage, message sending and other actions

      * "Google App Engine" - Google's own attempt

      * Heroku <https://www.heroku.com/> - Allow developers to build a
      microservices architecture based on the services they provide like
      the example they mentioned in this blog
      <https://devcenter.heroku.com/articles/event-driven-microservices-with-apache-kafka>

AUTHOR

    Deriv Group Services Ltd. DERIV@cpan.org

CONTRIBUTORS

      * Tom Molesworth TEAM@cpan.org

      * Paul Evans PEVANS@cpan.org

      * Eyad Arnabeh

      * Nael Alolwani

LICENSE

    Copyright Deriv Group Services Ltd 2020-2021. Licensed under the same
    terms as Perl itself.

