IPFS Weekly 5

by Richard Littauer, Andrew Chin on 2016-03-04

IPFS is a new hypermedia distribution protocol, addressed by content and identities, aiming to make the web faster, safer, and more open. In these posts, we highlight some of the development that has happened in the past week. For anyone looking to get involved, follow the embedded hyperlinks, search the wealth of information on GitHub or join us on IRC (#ipfs on the Freenode network).

If you would like to get this update as an email, sign up for our weekly newsletter!

This weekly covers the last month. Here are some of the highlights for the February sprints!

Highlight

Our friends and users at ConsenSys wrote an excellent "Introduction to IPFS", which starts with a less-technical preface, and then dives deep into a full explanation of how the IPFS object model works. It walks through multiple examples, including directory structures, version control systems, and blockchains. This is a great post to familiarize yourself with how the low level IPFS objects work, with graph visualizations and in-depth explanations. It also features a fantastic cover image! Thanks @ChrisLundkvist and @ConsenSysAndrew!

Updates

go-ipfs

  • @lgierth improved the automated docker image for go-ipfs, which is now named ipfs/go-ipfs. Give it a try: docker run -i --net=host ipfs/go-ipfs. The new image is built automatically for every commit, and has automatic tags for the coming releases. It's also a few MB smaller, and doesn't require mounting a volume for the IPFS repository anymore. Instead, if there's no volume mounted, it'll generate an ephemeral identity and configuration which will be lost when the container exits. This is ideal for testing or just trying something out real quick. Images for go-ipfs 0.3.x are not provided, because the Dockerfile shipped with these version isn't suitable for Docker Hub automatic builds.
  • @noffle, @whyrusleeping and @chriscool have been improving the test suite, fixing flaky tests which used to fail inaccurately. This will let us merge PRs faster, as CI should correctly categorize success and failure.
  • @noffle also made progress toward getting go-ipfs compatible with vanilla go get installation. Hope to have more tangible results to show by end of the month.
  • @RichardLitt made a lot of documentation fixes to the CLI while working on the IPFS HTTP API.
  • @chriscool rewrote the build documentation for Windows, which enabled @slothbag to get Windows builds working again with minimal fuss.
  • @lgierth posted a writeup on the migration from 0.3.x to 0.4.0.
  • @noffle improved FUSE connection termination.

GX

@whyrusleeping shipped a PR that introduces the use of a tool called gx, for vendoring our project dependencies. Previously we used godeps, and saved all of the code required indirectly to build ipfs in the go-ipfs repository itself. This was very difficult to work with for a few different reasons, first off, it made the size of the repository bloat way more than the original size of our codebase, causing clones to take longer, and making CI slower all around. Second, updating these dependencies was a hassle: this was partially the fault of go's poor package management choices and partially that we found godeps UX to be unfriendly. To solve the problem, @whyrusleeping create gx. Gx is a package management tool based on ipfs. Package references are all hashes linked in a merkletree, and resolving all the dependencies of a given project is as easy as an ipfs fetch. Now that we are using gx, the main go-ipfs repo is much smaller, dependencies can be easily fetched and installed (and shared across projects), and we also get to essentially bootstrap ipfs with ipfs.

From the average users perspective, there are a few small changes; go get is no longer a viable way to install ipfs, and users will now need to run make install as is common on other large golang projects (docker and kubernetes among them). You can read more about gx over here at its repo: github.com/whyrusleeping/gx and about gx-go (the subtool of gx specifically for go) here: github.com/whyrusleeping/gx-go.

@RichardLitt improved the gx README document to better explain its goals, and help people get started. Take a look and give us feedback through an issue if you think anything could be improved.

ipns-pub

@whyrusleeping wrote a tool called ipns-pub to allow people to publish IPNS entries to the network without actually running a node. You can generate keypairs with ipfs-key and then use those keys to publish any ipfs path you like. Be aware though that entries published with this tool expire every 24 hours, so to keep them alive on the network, the tool has a --daemon option that will republish your entry every twelve hours automatically.

specs

The IPLD spec was merged, after months of thoughtful design. The bulk of the work was achieved by @mildred and @jbenet, with lots of comments and design opinions from many other contributors. The IPLD spec. The "thin-waist" Merkle DAG format, defines merkle-links, -dags, and -paths, as well as the IPLD Data model and formats. In short: JSON documents with named merkle-links that can be traversed. Stay tuned for more information in the future.

http-api-spec

@RichardLitt finished logging all existing ipfs commands for the HTTP API Spec. This means that if you have any questions about how the HTTP API should work, you can see them either in master or as open PRs to that issue. If you are interested in how the HTTP API should work, or have any specific questions, please see the current version, and explore the open PRs on that repo.

distributions

dist.ipfs.io almost has signed releases thanks to @dignifiedquire. This will be coming soon.

js-ipfs

Thanks to @diasdavid, the DAG object manipulation commands now work, with tests and all. As well, thanks to @dignifiedquire's efforts, the js-ipfs API now returns promises if there isn't a specified callback, allowing both major methods used by the Javascript community to work equally well.

registry-mirror

@diasdavid improved registry-mirror performance and robustness by removing the dependency on registry-static. The few necessary parts were copied in. This is a huge step in reliability and performance cloning the registry with registry-mirror.

station

@dignifiedquire fixed drag and drop file uploading, and some dependency issues. So go ahead and try it out.

ipfs-geoip

@dignifiedquire rewrote the generation script and cleaned up the code so that now the data is consistently reproducible and fully stored on IPFS. This ensures that geoip lookups over IPFS will work in all future releases.

fs-repo-migrations

@chriscool improved important tests for fs-repo-migrations -- tests verify more edge cases when migrating forward and backward, through various sample workloads.

ipfs-hyperlog

@noffle built ipfs-hyperlog, and ipfs-compatible fork of hyperlog, a DAG that replicates based on scuttlebutt logs and causal linking. ipfs-hyperlog is a drop-in replacement for @mafintosh's hyperlog. Its key difference is that it creates a Merkle DAG that is binary compatible with IPFS objects. This means any node of any DAG built using ipfs-hyperlog can be replicated to and from the IPFS network as well!

Logo

@Kubuxu worked on a new IPFS logo. Check it out.

Community

  • If you have any suggestions for conferences, there is a new issue where you can submit your proposals. The IPFS community will see if we can get to that conference and have a presence there.
  • We now have a list of community events for IPFS on the ipfs/community README. Do you have any that you'd like to add? Check out past events, too.
  • @RichardLitt proposed using the new GitHub Templates for IPFS. What do you think about this idea? Chime in here.

IPFS In The Wild

  • Marmot Image Checker: This takes an image, asks the Google Cloud Vision API for three descriptions of the image, and compares those descriptions to a chosen list of words. If there is match, the image is added to the toadserver, a smart contract and IPFS-Backed download server. From Twitter
  • asciienema now allows playback on IPFS. From Twitter

Contributors

Across the entire IPFS GitHub organization, the following people have committed code, created issues, or made a comment on GitHub between February 1st (noon, GMT) and February 29th. We're autogenerating this list using this tool and this other tool, so please let us know if your name isn't here.

This newsletter is also a community effort. If you have cool things to share for the next weekly, drop a comment about it in [the weekly repo](//github.com/ipfs/weekly! The more people mention items they want to see in the weekly there, the easier it is to make this and send it out.

  • Richard Littauer and Andrew Chin

Submit feedback about this issue here, or send us feedback about the IPFS Weekly in general.