Archive for July, 2016

Low maintenance blog

Posted: July 28, 2016 in Work Stuff

This uses a version of Ghost in a Docker container that then serves up an external folder as the blog root

It allows Ghost to do what it does best – simple blogging tools and site management while using external scripts to ensure backup of the content.

If we lose the blog server, it is very easy to rapidly redeploy the blog and get back up again.

Start the blog

We will create the basic blog and associate it with a remote repo for backup. This assumes that Docker is installed in the host machine!

1. create a remote repo that will be used to store the blog
2. check out the repo to the box that will serve the blog e.g. /home/blog/content
3. start up a Docker for the Ghost blog

docker run -d -p 80:2368 -v /home/blog/content:/var/lib/ghost -e "NODE_ENV=production" --name myblog ghost:latest

This starts the Docker and tells it to map the Docker Ghost dir (/var/lib/ghost) to the real directory (/home/blog/content). It also maps traffic from port 80 through to the normal Ghost port of 2368 and runs everything in daemon mode.

Backup the blog

We can now install a script to monitor the blog and push changes up to the remote repo. This assumes that either username/password can be used in the https git calls or that the appropriate .ssh keys have been setup and the repo uses ssh to connect.

4. install gitwatch and it’s dependencies (git & inotify-tools)
5. point gitwatch at the blog directory

gitwatch -s 300 -r git@github.com:username/your-repo.git /home/blog/content/

This waits 300 seconds after a change has been detected in /home/blog/content/ and then pushes it out to the specified repo.

6. the above command will need to be run in the background either using screen or nohup to make sure it runs all the time.

Belt & Braces

We could also install a script called buster to automatically convert a Ghost blog to static pages compatible with Jekyll and/or GitHub pages. This would allow us to publish the blog on other platforms if we decided that Ghost wasn’t our platform of choice anymore.

As buster generates a static folder in the blog dir, it will also be pushed back to the remote repo if anything changes.

Redeploy

If the worst happens, the Docker can easily be restarted with the same command as above. If the entire box is lost, the repo can be checked out on to a new box and the Docker started as normal. The only thing that may need outside assistance is ensuring that the domain name points to the new machine.

Contracts on the BlockChain

Posted: July 20, 2016 in Work Stuff

Introduction

Initial investigations into using the Ethereum BlockChain were targeted towards the use of Smart Contracts.

These Smart Contracts have the potential to help my employer manage entitlements and digital/asset rights without requiring a single point of approval.

The Contract

As a paper based exercise prior to trying to convert the theory into practise, the following three contract types have been considered.

  1. Generic shop
  2. Asset based
  3. Customer based

As Ethereum contracts come with basic storage capabilities it allows us to store data in a way defined by us within the contact. This can be anything from simple key-value pairs through to complex structures.

The contract would then store the information about the transaction depending on the contract type – the customer, the asset and any access rights associated with the transaction in its storage.

Playback would depend on checking that the user requesting the asset on the current device meets any access rights. Success would allow the player to continue, failure would cause playback to stop with an appropriate message.

In the case of the generic shop contract, the assumption is that there would be a single version of this contract on the BlockChain that then handles the duties for the entire marketplace.

For asset based contracts, each asset could have its own contract deployed on release to the marketplace. Each transaction for that asset would be stored in the contract.

For customer based contracts, each customer would have an account contract deployed on signup that then contained a list of available assets.

In all three scenarios, the assumption is that contract storage would be used to store the relevant information. However, if the contact is part of a private BlockChain, it could add blocks to that BlockChain rather than using contract storage.

In either case, storing information back to the BlockChain – either in contract storage or as blocks – has a cost associated with it that is proportional to the amount of data to be stored.

Each version has its benefits and drawbacks as outlined below.

Generic Shop Contracts

A generic contract like this acts as the central store in the asset lifecycle. The system would allow Customers to sign in with their account address (allowing access to the customer’s wallet) and then handle the transaction between the buyer and seller of the asset.

This would then allow for both open purchases, limited purchases (time, device, resolution and/or playback limits) as well as rentals or downloads.

Pros
1. Potential for single contract for all transactions.
2. If necessary, each Provider could extend the basic contract via inheritance to enable additional access rights that are relevant to them.

Cons
1. Would require extensive storage within the contact.
2. Speed of searching the storage to find a match could be detrimental to playback.
3. Single point of vulnerability – contract would need to be complex to handle all provider requirements.
4. Adding new requirements or providers could be problematic.

Asset Based Contracts

These contracts are associated with a specific asset or title. It would allow for a contract that holds the basic asset information that can then be extended by using inheritance to add contracts for specific versions or use-cases.

The provider can modify the contracts to better reflect their actual requirements on a per asset basis. New or popular titles can have tighter requirements. As assets get older, new contracts can be issued that reflect the changes.

 Star Wars
    |\
    |  \___ Star Wars UHD
    |   \
    |    \___ Star Wars HD
    |   
Star Wars SD        

Potentially, the entire contract could be updated allowing existing owners to benefit without changing individuals. For example, if the SD version was no longer available, all the current SD owners could be migrated across to the HD version without any action on their part.

The sale of the asset would need to be registered within the storage of the associated contract.

Playback would require searching the relevant asset contract for permission to proceed.

Pros
1. Allows for fine control over assets.
2. Can deploy multiple contracts & sub-contracts to limit risk of exposure or speed up validation searches.
3. Can revoke entire contract or sub-contract if asset security breached.

Cons
1. Lots of storage if asset is popular.
2. Speed of searching through storage.
3. More management for providers.

Customer Based Contracts

These contracts are associated with a specific customer. It allows for a contract that holds basic information about the customer. If necessary, the providers could extend this info via sub-contracts to ensure that they had information relevant to their access requirements.

 Customer Account
   |\ 
   |  \___ Studio 1 Account
   |   
  Studio 2 Account

Pros
1. All customer’s assets in a single storage tree.
2. Quicker rights searches.
3. Single master account for all providers.
4. Potentially easier to validate all assets on start-up allowing for status messages/alerts and removal of expired assets.

Cons
1. Customer loses the account, loses all the content.
2. Assumes all providers use the same system.
3. Account breached, all content exposed.

Ethereum – A Quick Overview

Posted: July 18, 2016 in Work Stuff

Introduction

Ethereum is a technology that extends the concept of the secure BlockChain (made popular by BitCoin) by the addition of smart contracts. These contracts allow for facilitating, enforcing and validating of agreements between two or more parties.

The idea is that the smart contracts are written to the BlockChain and then deployed into a virtual machine that then runs inputs against the contracts and returns results. The contracts are written using one of two Ethereum specific languages and then compiled into a bytecode version for use by the Ethereum ecosystem in the form of DApps.

Once deployed to the BlockChain the contracts cannot be changed, but can be used by anyone.

The Idea

The initial concept behind investigating the use of smart contracts in Ethereum was the implementation of a marketplace for video assets. By using a smart contract, we could potentially enable both digital and access rights management for an asset, without the need for third-parties to provide ‘trusted services’.

This would allow for Studios to sell niche or older assets directly to the customers without the costs associated with going through normal video sale channels.


The Issues

Investigations into using Ethereum have run into issues that have effectively prevented us from building any form of working anything, let alone an actual prototype.

The platform was evolving fairly rapidly, with all the associated issues that this causes – out of date tutorials and documentation, broken tools, fragmented development environments and frameworks. Given the scale of the platform, this fragmentation is more obvious and problematic

From A 101 Noob Intro to Programming Smart Contracts on Ethereum

A development cycle in Ethereum would be to develop a contract locally with testing to ensure it works. Then you can either deploy to the Ethereum TestNet or to a private BlockChain in your own network that you have created. This would allow access by multiple users simultaneously to ensure that the contract still behaved as expected. Finally, the contract would be deployed into the real world for use by the DApp clients.

Tools

To build a contract you need a set of tools that work – even at this early stage we would expect the ability to write a contract and debug it without significant issues.

However, the toolset to build and deploy contracts is not complete. Issues include the deprecation of key components without working replacements. Tools such as AlethZero that supposedly allowed you to deploy contracts and interact with them are being replaced by a new IDE called Mix.

Mix needs to be built from source, but fails to build on most occasions. Once it does build, it is unstable and exits without warning when trying to write and test contracts. Mix is apparently the only tool that allows you to see inside a BlockChain for debugging. Without this ability, it becomes difficult to properly test contracts and the libraries trying to interact with them.

While AlethZero still exists, it is not updated and now seems to be missing functionality caused by the platform evolving, limiting its usefulness.

The core Ethereum ecosystem is written in Go and C++, with two distinct languages created for writing contracts – Serpent (with a syntax similar to Python) and Solidity (with a syntax similar to JavaScript). Both languages need to be compiled to bytecode (Abi) prior to use.

Converting to bytecode requires a compiler that also needs to be built from source – although this is a more reliable process than building Mix, it is still unpredictable. Tools such as Mix have a compiler built in, but as they are unstable, getting a contract written in another editor and then converting it is an awkward process.

Frameworks

There are two significant frameworks written by third-parties that attempt to make the contract creation process much less painful – Truffle and Embark. Both are NodeJS based apps that allow you to undertake some contract development. They allow for the creation of a contract, compilation to a JavaScript library and deployment as part of a web-based app. However neither allows you to actually see the resulting BlockChain and its contents for debugging and neither are of any use if you want to write an app that is not HTML based.

Both frameworks also contain bugs that have caused issues when trying to build contracts.


Summary

While the idea behind Ethereum and the use of Smart Contracts has a huge amount of potential, it is currently hampered by a poor tool set and development environment. This situation is not likely to improve in the near future as the platform is frozen while the community tries to find a solution to the problems caused by the loss of almost $53 million in funds from the DAO core system by a hack.

This hack also served to underline some significant flaws in the virtual machine that runs everything and the languages that have been built upon it. It appears that changes to the underlying VM will be needed to remove the revealed security flaws. Developing a new VM would mean rebuilding the BlockChain from scratch on the new VM, something that is unlikely to happen given the amount of money currently locked into the Ethereum ecosystem.

The languages available, although being presented as a language especially tailored for Smart Contracts, are not much different than regular programming languages (in fact, they are Turing complete). This means that everything written in them is entirely procedural and that no notions, terminology or vocabulary of high-level concepts of contracting can be used in the contracts’ code. While this might not seem important at first sight, it actually entails two big problems:

1. The procedural code can potentially be too complex to understand or verify prior to execution. This is one of the main reasons that the DAO code, even after being audited twice, had important vulnerability issues causing the problems aforementioned.

2. Contracts cannot be properly monitored or formally checked without the use of high-level contracting notions (fulfilment, violation, parties, clauses) at the language level. This is widely accepted in the field of the research of electronic contracting but the Ethereum developers have completely ignored it.

Given the weakness and instability in the platform and its toolset, we feel that further investigations in this technology needs to wait for either an alternative platform to emerge or for Ethereum to address these issues and create a more stable toolset.