Knomaton

Data

If you go back to the first single-cell form of life, it clearly possessed the capacity to receive, to utilize, to store, to transform, and to transmit information.
-- Dee Hock

Most of the informatin we use is in form of a list of some objects, e.g list of emails, contact list in mobile phone, events in calendar, items in eshop and so on. We usually use common operations to manipulate these data:

  • CRUD operations
    • Create
    • Read
    • Update
    • Delete
  • Querying
    • Sort
    • Filter
    • Aggregate
  • Transformation

Most applications provide only some fraction of above features, because with current database systems they usually require extra effort by developer (which requires extra time and money, so vendors do not offer complete support).

Warehouse.js and RQL

An ongoing effort to improve this situation includes implementation of Warehouse.js library.

Warehouse.js is a data storage layer for node.js and browser providing unified API for many supported storage engines (MongoDB, SQL, ElasticSearch, REST, in-memory and others). API provides functionality for CRUD operations and querying using Resource Query Language (RQL). Transformations can be done using MapReduce functionality.

Recline.js

For non-programming users, graphical widgets for data manipulation, querying and visualisation must be developed. One such project is Recline.js developed by Open Knowledge Foundation.

Databases

There are many kinds of database architectures. See Wikipedia article or this article comparing following architectures:

  • RDBMS, or Row-Oriented database
  • Semantic net
  • Entity-Attribute-Value
  • Column-Oriented databases
  • Correlation databases
  • Hierarchical model, Network model, Navigational database
  • Object-Oriented databases
  • Document based databases
  • Triplestores

Data management software

DabbleDB

One product that came close to my vision was DabbleDB wikipedia link. However its development was discontinued and it is not available anymore.

Others

Commercial proprietary solutions exist which implement more or less of the vision. Examples are FileMaker, Quickbase or Zoho Creator.

There are also interesting ideas in discontinued product Lotus Agenda wikipedia link.

Data manipulation tools

There exist some useful tools for manipulating and cleaning large datasets.