[LRUG] CMS for bespoke API

Wed Sep 11 03:17:57 PDT 2013

A universal CMS based on JSON is in the making. It is however much more
low-level and  ambitious than what Ian would need. Check out
http://camlistore.org/docs/overview

I'm very excited about this technology - it is being primarily driven by
Brad Fitzpatrick (http://bradfitz.com/), who has a history of building
scalable content management solutions (LiveJournal, Memcached) amongst
others like OpenID and PubSubHubbub.

The basic component in Camlistore is a blob, which is just dumb data
without any identifiying metadata. The blobs are uniquely addressed by
using sha1 keys called blobrefs. The basic operations are `get(blobref)`,
`put(blobref, blob)` and `enumerate(after, limit)`.

The lower-most layer of Camlistore is this dumb blob server.  On top of
this we have a schema layer. This is where the metadata about the blobs are
stored. All metadata is stored as JSON - here is an example of how a file
would look like:

{"camliVersion": 1,
  "camliType": "file",
  "fileName": "foo.dat",
  "unixPermission": "0644",
  ...,
  "size": 6000133,
  "contentParts": [
    {"blobRef": "sha1-...dead", "size": 111},
    {"blobRef": "sha1-...beef", "size": 5000000, "offset": 492 },
    {"size": 1000000},
    {"blobRef": "digalg-blobref", "size": 22},
   ]
}

The interesting thing is that the schema files are also saved as blobs, and
referred to using blobrefs. It is content addressable blobs all the way
down. I borrowed the schema file description from
https://docs.google.com/presentation/d/1otsEBBmOYoPY6T7p4YtM0ysX1vUG3amZQAykr9jZZgk/present#slide=id.i0-
it is a short presentation by the authors that covers the architecture
in
good detail.

Camlistore borrows many things from Git - this includes the philosophy of
having small low-level building blocks on top of which complex
infrastructure can be built, using sha1 addressable blobs and preserving
immutability by using diffs for mutation.

It comes with an indexing server, a web frontend for managing picture
uploads, a FUSE driver to mount file and directory schema entries, and
support for syncing to the cloud. It will work with both small and large
files, and is especially optimized for things like VM images. Large files
are typically split into many small blobs, and thus localized mutation will
affect only a small number of blobs. Also since the blobs are quite dumb,
we can distribute and sync them across systems without worrying about their
higher meaning.

The whole project is written in GoLang and there is active development
happening. The overriding goal of the project is to have a storage server
where all our digital data can be dumped into, on top of which complex
content management systems can be built. This project is part of the
Indieweb movement where privacy and control of our data are fundamental
principles. (http://www.wired.com/wiredenterprise/2013/08/indie-web/)

The project is in active development, and even though I wouldn't recommend
it for any kind of serious use yet, it should mature soon and would be
something to watch out for.

--
Jasim A Basheer -- http://nilenso.com
http://twitter.com/jasim_ab

On Wed, Sep 11, 2013 at 3:07 PM, Riccardo Tacconi <rtacconi at gmail.com>wrote:

> Hi,
>
> As you said perhaps it does not exist and I think it doesn't. They should
> realise that they need a bespoke CMS. I would suggest to create  an open
> source project and make the CMS public so it could become, in the near
> future, an Open source JSON CMS :-)
>
>
> On 11 September 2013 10:31, Ian Kynnersley <iankynnersley at gmail.com>wrote:
>
>> Thanks for all the useful responses. I'll definitely look into Active
>> Model Serializer.
>>
>> Most of these replies rely on me creating a bespoke model structure in
>> code which is what I'm trying to avoid. Parse and Helios look interesting
>> but don't give me the admin interface that I need to allow totally
>> non-techie users to be able to manage the content themselves.
>>
>> What I'm after would be a library that I can install in my app (or
>> something hosted like Parse / Helios) that instantly gives me my CMS.
>> Something a bit like Radiant maybe. I would then want to create my model
>> structure within that CMS such that admin users could add, remove and
>> update information easily. However, unlike a normal CMS which would render
>> the output as a website with some templates to alter the look and feel, I
>> want it to provide a JSON API.
>>
>> The reason for all of this is that ownership of this app will be handed
>> over to a different company and they are (rightly) nervous about taking on
>> a completely bespoke application rather than a customised version of an
>> off-the-shelf product.
>>
>> Perhaps I am looking for something that doesn't exist so your thoughts
>> are all very useful.
>>
>> Thanks again
>> Ian
>>
>>
>> On Wed, Sep 11, 2013 at 3:20 AM, Tejas Dinkar <tejasdinkar at gmail.com>wrote:
>>
>>>  On Sep-11-2013, at 7:02 AM, Nick Novitski <nicknovitski at gmail.com>
>>> wrote:
>>>
>>>  There's also https://github.com/rails-api/active_model_serializers
>>>
>>>
>>> +1 on Active Model Serializer.
>>>
>>> Benefits:
>>> 1) Easily testable, and OO
>>> 2) Has access to current_user (or any other controller method via
>>> `scope'). This makes it possible for you to serialize based on who has
>>> requested the object
>>> 3) Works seamlessly with respond_with
>>> 4) If you have the same relation object multiple times on your object
>>> (say user can be created_by, updated_by, etc…) then you have a way to just
>>> `embed' the user once.
>>>
>>> Also, +1 on rails-api gem as well :-)
>>> --
>>> Tejas Dinkar
>>> http://www.nilenso.com
>>> Nilenso Software (formerly C42 Engineering)
>>>
>>>
>>> _______________________________________________
>>> Chat mailing list
>>> Chat at lists.lrug.org
>>> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>>>
>>>
>>
>>
>> --
>>
>> Ian Kynnersley
>> http://iankynnersley.co.uk | +44 (0) 7973 420 829 |
>> http://twitter.com/kpopper
>>
>> _______________________________________________
>> Chat mailing list
>> Chat at lists.lrug.org
>> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>>
>>
>
>
> --
> Riccardo Tacconi
> Ruby on Rails and PHP development - System Administration
> VIRTUELOGIC LIMITED <http://www.virtuelogic.net/>
>
> http://github.com/rtacconi
> http://riccardotacconi.blogspot.com
> http://twitter.com/rtacconi
>
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>
>

-- 
Jasim A Basheer -- http://twitter.com/jasim_ab
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20130911/57ad6493/attachment-0003.html>