Technology

Evolution

MindTouch initially developed MindTouch Deki from a MediaWiki fork that was adapted for enterprise usage and heavily tested for usability. The initial release of MindTouch Deki, "Gooseberry", was a point application and was made public on July 2006. This version shared a significant amount of the MediaWiki codebase except MindTouch had converted the entire system to be XML-based, added a WYSIWYG editor, adapted Lucene to provide enterprise search capabilities across pages and file attachments, added the ability to organize and permission access using hierarchies, and provided an entirely new user interface. This early release was made public at Sourceforge.net and comparable in feature set to other enterprise ready wiki point applications available today such as Twiki or Atlassian's Confluence. The early "Gooseberry" version of MindTouch Deki was a way for MindTouch to get to market quickly with an enterprise ready product in order to receive community feedback that would guide development of the new platform, which had been in concurrent development since the conception of MindTouch. This new work began with building MindTouch Dream as the foundation for MindTouch Deki version "Hayes", the distributed application platform that was released in July 2007.

As of the MindTouch Deki "Hayes" release, in July of 2007, the architecture of MindTouch Deki is a composition of loosely coupled web services that are orchestrated by MindTouch Dream. The majority of the business logic of MindTouch Deki is implemented in C# and MindTouch deploys this on Linux using Mono or Windows with .NET. On top of this services layer are several interfaces, or clients if you will. The most robust interface is a PHP client. All communication between the clients and the services layer, commonly referred to as MindTouch Deki API, is done over HTTP using standard HTTP verbs: GET, PUT, POST, etc. Additional interfaces to the services layer exist, including a Win Form client and an Microsoft Outlook Connector. More interfaces are being developed by MindTouch and the community. MindTouch Deki uses these interfaces and connectors to act as a switching board between multiple applications and data stores; by far, most interest from users has been in this area.

MindTouch Dream

Dream is a REST microserver for web services and a toolset for easing the complexity of distributed heterogeneous programming.  This kernel is the foundation for MindTouch Deki.

MindTouch Dream is a service-oriented, or distributed, application framework that is the foundation of MindTouch’s technology. It is built on many years of experience and research into distributed computing, web-services and self-organizing systems. Its design enables faster and more reliable development of rich, extensible, and scalable N-tier applications. Dream is implemented in C# and executes on Mono or .NET. Dream's libraries facilitate interaction with other REST-based web-services, easy manipulation of XML data, and definitions for creating new Dream services. The core libraries are provided for .NET, but auxiliary libraries are available for PHP and JavaScript to simplify integration. The Dream runtime environment manages services initializing, message routing, and workload distribution. The Dream runtime uses an efficient lightweight-threading model that scales nearly linearly with the number of CPU cores in a computer. The core Dream services provide common building blocks such as event dispatchers, distributed directories, logging, and service inspectors. All of these capabilities are packaged in a very compact and easy to learn framework that is there to assist developers in producing better quality services faster.

Dream provides a new way to develop software that is more conducive to large scale enterprise web applications. With Dream, web services are similar to objects in that it gets instantiated and has entry points.  These entry points are called service features or features for short.  A feature is an interaction point for another service. Since all Dream services are RESTful, features interact through standard HTTP verbs such as GET, POST, PUT, and DELETE.  The design of the Dream runtime allows a service author to assume an idealized world where everything that the service comes into contact with is accessed through web-requests.  The Dream library provides classes with simple interfaces to enable these interactions.  The benefit of the idealized world assumption is that it reduces the number of cases that a developer needs to be aware of.  Note that an idealized world doesn't mean that everything is abstracted away.  Quite the contrary.  Instead, everything fits one view and this view makes the following assumptions:

1. Other services may become unavailable at any time
2. Resources must be managed through timeouts (i.e. time-to-live/TTL)
3. Everything runs concurrently 

These basic tenants permeate MindTouch Deki.

Traditional Application Architecture

MindTouch Deki's architecture is dramatically different from traditional software applications. To explain this, let's compare it to how most software is architected. Traditional software applications are developed in a monolithic fashion. For example, when a web application is developed it looks like the image shown in Figure 1.

Figure 1

Traditional Software Architecture

The application in Figure 1 is built in a specific programming language and requires a specific operating system. When an engineer needs to extend the application, the engineer must use the same programming language as the original implementer to create new extensions and these must be installed locally on the same operating system. In short, extending applications requires the engineer to build inside the same monolithic environment using the same components that the application uses, regardless of the toolset the engineer is already familiar with. Furthermore, integrating other systems and services is unnecessarily difficult because it requires a deep understanding of the monolith, similar styles and disciplines in engineering, and if a break is introduced to the stack, the entire monolith can be rendered inoperable.

This monolithic approach to software development has not changed in decades. In today's connected world, this model does not lend itself to capturing the capabilities of applications that live inherently in the network. It is inefficient because even when an application has a network accessible API -- and most don't -- it only behaves as an end point, a behavioral silo limited to the confines of its own monolithic architecture. These point applications are incapable of realizing the latent benefits of the network fabric that exist in organizations and across the world. When the application exists in the network instead, many desirous

Figure 2

MindTouch Deki: A Distributed Heterogeneous System

effects are realized, including improved scaling, ease of extensibility, ease of integration, incremental development costs, better use of engineering resources, and more...

MindTouch Application Architecture

MindTouch takes a very different approach to software development. 

Figure 2 provides a graphical depiction of MindTouch Deki's software architecture. MindTouch Deki is a composition of loosely coupled heterogeneous services that exist across networks. They share their behavior in a distributed and concurrent manner through the MindTouch Deki API and Dream (Dream is omitted for simplicity, see Figure 3). New functionality can be added in any programming language on any operating system executing on any server that communicates over any web protocol. In short, the application lives in the network. Rather than requiring adherence to a particular interface in a particular language, such as Java or C#, extensions only require adherence to a stateless web protocol. In essence, this is the wire implementation and the benefits are far reaching.

Let's take a more detailed look at the architecture of MindTouch Deki. 

Figure 3

Detailed view of MindTouch Deki's architecture

As mentioned earlier, MindTouch Deki is a wiki interface to a distributed application platform. Note the red box labeled "wiki interface". This is the user-interface client that communicates to the MindTouch Deki API (red box) over HTTP/S. The MindTouch Deki API uses Dream (yellow box) to federate and orchestrate functionality from other services, applications, and data stores, enabling MindTouch Deki to be used as an information bus across all of them. In essence, MindTouch Deki is the connective tissue within the enterprise.

Application behavior and data is surfaced to end users with the help of Dream and through the MindTouch Deki API. End users can create dynamic reports of data, composite applications -- sometimes called situational applications -- or data mashups. A site administrator approves the connections to applications, data, and services through an easy to use interface. End users can easily expose data and functionality to others without giving direct access to the applications or databases.

Concurrency

Figure 4

Traditional software execution completion is the sum of all operations
T = t₁ + t₂ + t₃

One of the many strengths of Dream, and therefore MindTouch Deki, is the ability to process and communicate concurrently. To explain this let's first examine how traditional software applications process and communicate.

In Figure 4 you can see that as an application executes it does so serially. This means each request is dependent upon the completion of a previous requests. The system holds until all requests are completed. To illustrate this let's assume that above we have a user who is accessing an application that sends a request to a CRM system, a database, and then stores information. With traditional applications, each of these requests are completed before the next begins, the time for completion is the sum of all activities, or it can be written as T = t₁ + t₂ + t₃, where T is time and t_n is the time it takes for individual requests.

With Dream this is handled very differently.

Figure 5 shows how Dream, and therefore MindTouch Deki, handles execution. Dream issues requests concurrently, which means the application and end user only wait for the longest operation to complete rather than waiting for the sum of all operations. In a network native application where latencies are common, this makes a very big difference. For example, the user sends a request to MindTouch Deki, it sends requests to the CRM system, a database, and asks for a file to be stored

Figure 5

MindTouch Dream (and MindTouch Deki) execution completion is equal to the longest operation
T = max(t₁ , t₂)

simultaneously and concurrently. Because the file being saved to the storage service doesn't require any further interaction from MindTouch Deki or the user this can be done in the background without slowing the completion of the operation. This achieves dramatically improved system performance and can be written T = max(t₁ , t₂), where T is time and t_n is the time it takes for individual activities. As you can see, t₃ is omitted, because it does not contribute to the user's wait time.

With Dream and MindTouch Deki, the system's ability to distribute itself and execute concurrently provides a significantly better performing platform to build on.

Extending the Platform

Extending MindTouch Deki requires no knowledge of the internal workings of the platform.  All interactions are achieved through a stateless web protocol that does not impose any specific programming language, operating system, or deployment location.  New capabilities can be added along several dimensions:

• embedding content from other applications or services
• authenticating users against other systems
• storing information in the cloud or in the network
• searching within the platform or across the enterprise
• visualization diverse content types

Similary, the MindTouch Deki user interface is decoupled from the MindTouch Deki API, meaning that the capabilities of MindTouch Deki can be embedded into existing applications or access from specialized front-ends on the web or on the desktop.  MindTouch provides two popular desktop front-ends to publish emails from Microsoft Outlook and drag-and-drop files and folder inside Microsoft Windows.