IPFS Weekly 7

by Richard Littauer, Andrew Chin on 2016-03-17

IPFS is a new hypermedia distribution protocol, addressed by content and identities, aiming to make the web faster, safer, and more open. In these posts, we highlight some of the development that has happened in the past week. For anyone looking to get involved, follow the embedded hyperlinks, search the wealth of information on GitHub or join us on IRC (#ipfs on the Freenode network).

If you would like to get this update as an email, sign up for our weekly newsletter!

Here are some of the highlights for the second week of March:

Updates

Orbit

Orbit, a distributed, peer-to-peer chat application built on IPFS, is back in active development and going through a major code base overhaul. It now uses orbit-db as it’s database layer. orbit-db is a KV-store and Event Log on top of IPFS which allows developers to use IPFS as a database. Last week orbit-db got big performance and stability improvements and is now using CRDTs for eventual consistency.

Go-IPFS

A nasty bug in our networking code was fixed this week. An issue in yamux (our primary stream multiplexer) would cause code to hang when opening a new stream if there were too many in flight stream opens. As a result of this, large file transfers (ipfs refs -r, ipfs get, and so on) would hang.

IPFS 0.4.0 is now very close to shipping, we ran @dignifiedquire’s randor tool quite extensively and are more confident in the repo operations that have changed since 0.3.11. The release now has a somewhat short checklist of things that are blocking the official release.

As a small side project, @whyrusleeping started benchmarking each of our possible datastore implementations with ds-bench. The results will help us improve the performance of our storage moving forward. On that same topic, he started an experiment in new and exciting datastores and wrote a datastore implementation that can be backed by a sql database. @whyrusleeping then spun up a postgresql database (from docker) and ran the benchmarks against it, without any sort of tuning the initial performance metrics showed that it was around three time as fast as our current on-disk datastore. (Note: the metrics are simply testing the performance of writing random 256k binary blobs to each datastore).

js-multiaddr

The js-multiaddr implementation was upgraded, now that it now matches go-ipfs features. What this means is that js-multiaddr now supports protocols that need to be encoded with varints. A varint is a integer that uses only the number of bytes it needs to be described by using a continuation bit (more details). Before this addition, we were not able to declare http, websockets or even IPFS multiaddrs because these protocol have a code that doesn’t fit in a byte - now we can. You can check the encoding table here.

mafmt

mafmt, a module that @whyrusleeping wrote and which stands for multiaddr format, filters out different formats to use with certain transports. Thanks to @daviddias, this now does uTP and WebSockets validation, as well.

libp2p-swarm

The new libp2p-swarm has been released; to reiterate last week, the module has been overauled to have a new API, to be agnostic about which transport an application has to use, and also to have more tests. This also exposes more internal processes, which makes error checking and logging more useful. Libp2p will eventually help users traverse IPFS for any node which allows their desired transport, meaning that we can have a more connected and less brittle web.

Community

@jbenet was in Boston this week for consensus research discussions. While in Boston, @jbenet visited the Digital Currency Initiative at the MIT Media Lab, where he spoke about IPFS and its implications for blockchains, for digital publishing, and for access to information.

c-base/ipfs-ringpin is a new set of tools for setting up an IPFS “file pinning ring” between multiple parties. This can be used, for example, for hackerspaces to provide redundancy by pinning each other’s files. pipermerriam/ipfs-persistence-consortium and VictorBjelkholm/pincoop do similar things.

the-gred/jsjob-ethereum is an experiment at creating a computational market on top of Ethereum. Computational jobs are run in a JavaScript sandbox, and both input data, the algorithm to run, and the produced results are shared over IPFS.

@jbenet gave a talk titled Datastructures In and On IPFS at last year’s QCon SF. The video was released this week, so now you can all see it! The talk includes a typical introduction (skip if you know IPFS well), and then dives into datastructures, including Merkle Links, Mazieres links, how IPNS works, IPRS records, versioning, Keychain (PKI on IPFS), Persona (identity), and more.

datastructures-talk-img

The IPFS powered USB deaddrop at c-base was demonstrated at the LoganCIJ 2016 Symposium, in the Investigative Journalism conference. More pictures here. If you want to build your own IPFS USB deaddrops, check out the c-base project here: c-base/ipfs-deaddrop.

c-base-deaddrop

The IPFS Copenhagen Meetup organized by @NeoTeo had another meeting a few days ago. If you’re in the area, sign up at the Meetup.com page so you don’t miss the next one!

Several core IPFS developers will be in NYC from March 28th to April 8. We will be organizing an IPFS Meetup within that time frame. If you would like to come, please follow this github issue for the final details. If you would like to present, post in that issue.

Contributors

Across the entire IPFS GitHub organization, the following people have committed code, created issues, or made a comment on GitHub between March 7th (noon, GMT) and March 14th. We’re autogenerating this list using this tool and this other tool, so please let us know if your name isn’t here.

This newsletter is also a community effort. If you have cool things to share for the next weekly, drop a comment about it in the weekly sprint issue! The more people mention items they want to see in the weekly there, the easier it is to make this and send it out.

Thanks, and see you next week!

Submit feedback about this issue here, or send us feedback about the IPFS Weekly in general.