Container Component Types

The container has a large number of different types of plug-ins available to a developer, this is an outline of when and why to use the different kinds. To start creating components, see developing applications.

Starting at the top of the diagram, server instances are used for supporting external clients (if no suitable implementations of com.yahoo.jdisc.service.ServerProvider are available. A client may or may not be a network client, so it is related to the network layer. The network layer associates a Request with a response handler and routes it to the correct type of request handler (typically based on URI binding patterns).

If an application needs lightweight request-response processing using decomposition by a series of chained logical units, the processing framework is the correct family of components to use. The request will be routed from ProcessingHandler through one or more chains of Processor instances. The exact format of the output is customizable using a Renderer.

If doing search queries, SearchHandler will create a Query object, route that to the pertinent chain of Searcher instances, and associate the returned Result with the correct Renderer instance for optional customization of the output format.

The DocumentProcessingHandler is usually invoked from messagebus, and used for feeding documents into an index or storage. The incoming data is used to build a Document object, and this is then feed through a chain of DocumentProcessor instances.

If building an application with custom HTTP APIs, for instance arbitrary REST APIs, the easiest way is building a custom RequestHandler. This gets the Request, which is basically a set of key-value pairs, and returns a stream of arbitrary data back to the network.

Component is the basic building block in the container. A component may be injected into other components, including all the different request handlers, document processors and so on mentioned above, and will get a destructor like method invoked when no other component is using it any more.

Server

Servers are components implementing the com.yahoo.jdisc.service.ServerProvider interface, usually by subclassing com.yahoo.jdisc.service.AbstractServerProvider. Use this if developing support for a new type of clients, e.g. an unsupported network protocol.

A server, or server provider, in JDisc Context is a server protocol implementation. A server will usually be bound to some network port, and has the responsibility of creating JDisc requests from the incoming data and also transmitting the JDisc response back to the client in a pertinent manner. A server does not fetch content, that is the responsibility of a processor or request handler.

Request handlers

Request handlers are the easiest way to implement arbitrary REST services. The incoming request is a set of key-value pairs and protocol specific information. A request handler my fire off a request to e.g. the search handler, but what it will get back is opaque. If context sensitive result transformations are necessary, the request handler must be paired with one or more searchers, and if the output from the search needs to be customized, a renderer for the search result is the correct way of doing it (as opposed to trying to transform the search response in the request handler).

A request handler accepts a request, and returns a response. All applications will use a request handler, either by implementing their own, or indirectly by using the processing framework, as the processing handler is a request handler as well. A client provider is-a request handler.

Requests are modelled as key-value stores, as in many other systems, while responses are meta information (like HTTP status) and a binary stream of content (the ContentChannel API).

A content channel is the basic I/O building block of JDisc. It is conceptually similar to a stream, but the basic unit is the ByteBuffer instead of the native byte. A content channel will take ownership of a ByteBuffer written to it.

Response handlers

Response handlers are invoked by the request handlers after the client response is created. The correct response handler is chosen from URI pattern bindings. A Response is a more or less opaque data structure, so a response handler is more suited to e.g. tracking return codes than doing deep result inspections.

The response handler is passed along with the request, it is the link back to the external client, as it contains the infrastructure for passing binary data back to the network.

Processor instances

The processing framework is to be used for light-weight solutions where decomposition of the problem by partitioning the implementation into (a) chain(s) of processors is useful. Processors are invoked synchronously and the response is a tree of arbitrary data elements. Custom output formats is achieved by adding renderers.

Processing is a light-weight, low latency request-response framework built on decomposition by chaining. It is a multi-threaded API, each processor must be written thread safe.

As a request handler, a processor accepts a response and returns a response. There are two central differences in the processor API from request handler API. First of all, a processor may invoke other processors directly, as processors are organized in chains, like filters. Second, processing responses are structured trees, as opposed to a binary stream. This makes processors suited to both request and response transformation, as working on both the request and response is possible.

Renderers

Renderers are result formatters for the processing and search frameworks in the container. They usually render responses as JSON or XML, but arbitrary MIME types may be used, including binary formats. Renderers are subclasses of com.yahoo.processing.rendering.Renderer.

As a processing response is a tree, there must be some way to transform that to a binary representation. To support multiple output formats, the marshalling work is placed in the plug-in type Renderer. A default JSON renderer is available.

Searchers

Searchers are the only supported way of doing structured query and result transforms in a search context. If an application is performing searches and needs to look at single result documents (hits) or needs to add terms to the parsed query, searchers are the only way to do this robustly.

Document processors

Document processors are for documents what searchers are for queries. They can access and transform documents robustly and structured. If documents need some kind of transformations or annotations before reaching an index or storage, custom document processors are usually a better solution than manipulating the document feed.

Components

Usually, any container plug-in is a component. A component is usually a subclass of com.yahoo.component.AbstractComponent, though com.yahoo.component.Component is an interface. Components are the low-level wirings for using composition with injection in the container. If building complex searchers or document processors, shared resources or simply partitioning the code, is often needed. The Component is an object with a controlled lifecycle which may be injected into other components, such as searchers, renderers, document processors and request handlers

Binding

A binding is a pattern, or rule, matching a URI which is used to route incoming requests to the correct filter chain or request handler, and route outgoing requests to the correct client. For instance, the binding http://*/* would match any HTTP request, while http://*/processing would only match that specific path. If several bindings match, the most specific one is chosen.

Server binding A server binding is a rule for matching incoming requests to the correct request handler, basically the JDisc building block for implementing RESTful APIs
Client binding A client binding is a pattern which is used to match requests originating inside the container, e.g. when doing federation, to a client provider. That is, it is a rule which determines what code should handle a given outgoing request

Filter

A filter is a lightweight request checker. It may set some specific request property, or it may do security checking and simply block requests missing some mandatory property or header.

Client

Clients, or client providers, are implementations of clients for different protocols, or special rules for given protocols. When a JDisc application acts as a client, e.g. fetches a web page from another host, it is a client provider that handles the transaction. Bindings are used, as with request handlers and filters, to choose the correct client, matching protocol, server, etc, and then hands off the request to the client provider. There is no problem in using arbitrary other types of clients for external services in processors and request handlers.

Details

Clickable illustration:

DFD for JDisc container binding client binding client custom request handler processing processing request processing response processor processor processor JDisc request and response JDisc request and response JDisc request and response binding binding binding filter filter filter filter filter filter filter filter filter binding binding binding server server server