Another month, another EJTP release.

Continuing our monthly release cycle, EJTP v0.9.3 is out. And with some cool changes. And other stuff happened.

EJTP

There was a lot of refactoring this month, and a bit more still left to go.

  • We have entirely converted to unittest for the “brunt” of our test suite, only retaining doctests for actual documentation.
  • A bunch of cruft got ripped out - the repository is a lot cleaner and leaner.
  • In fact, structural organization in general is better, with things like ejforward moved into a new “ejtp.applications” package, and short/consistent names for the jack modules.
  • We use tox and Travis-CI for testing now.
  • The test suite is easier to set up on a development machine, no longer requiring you to install optional dependencies (but warning where they are not present, which helps you if you actually do want comprehensive local testing).
  • Elliptic curve cryptography is available as long as you have PyECC installed.
  • Compressed frames are now supported, with the zlib and bz2 algorithms. Adding more later would be trivial, if needed.
  • Frames completely rewritten from the ground up, in a way that provides better history metadata and extensibility.
  • New ejtp-crypto script, which allows for other languages (to which EJTP’s crypto parser has not been ported) to take advantage of its beautiful, consistent simplicity.
  • New ejtp.config module, for easily loading identities from files.
  • Full tests for all scripts.
  • Support for all Python versions 2.6-3.3.

With all the stuff that got in, a few things got squeezed out for lack of time/manpower:

  • Much like the frame refactor we did get in, we have refactors planned for the jack system and the router.
  • Extending the ejtpd protocol to include arbitrary code evaluation.
  • Undoing the crazy hacks we were forced into by our brief support of Python 2.5, making the code that much cleaner.
  • Standard protocols for both message splitting and establishing TCP-like high-efficiency streams (both depend on new middleware-friendly router architecture coming in the refactor).
  • IRC Jack.
  • TLS Jack.
  • Taking advantage of semantic versioning and having our 1.0 release.

In the next version, we’re going to primarily focus on finishing up the refactoring, then investing time and effort into polishing our scripts, tools, and applications. If we decide it’s stable enough to be our 1.0 release, it’ll be that, otherwise it will simply be v0.9.4.

CPS

I’ve been planning to overhaul CPS for awhile, I’ve just been putting it off. This month, though, I’m going to work to implement the following features, because DEJE’s DHT Bootstrap depends on all these things, and it makes sense to do them via CPS.

  • Make the entire library more modular, so that encryption/decryption is only one of several composable filters that can be conjoined into a single abstract datastore.
  • Multiplexer filter, which allows seamless and simple caching.
  • Kademlia support via Congress.
  • An alternative encryption filter where you provide an arbitrary encryptor during every storage or retrieval, rather than just at init time.

DEJE

I can finally get bootstrap working once I get the CPS stuff done. That’s where all the real complexity is. From there it’s just some fairly easy diagnostics stuff. Then I can finally get around to bounties!

Hardware design

Danry25 almost immediately provided an improved hardware design, and I’m really excited to assemble my first experimental tower. The only ingredient I still need to buy is the OmniTIK router, everything else is just sitting downstairs waiting to be assembled.

Various RI projects

I made a bit of progress here and there on multiple projects, inching forward to viability.

puppet-cjdns

The real pain point has been puppet-cjdns, a project I forked for straightforward CJDNS initialization on fresh servers. It didn’t work out of the box, it still doesn’t, but it’s getting there, and while it’s consuming all my “RI Business” time, I am getting close and closer to a working product.

Never try to build Augeas from source.

puppet-openvpn

This is a fork that I’m going to need to put much less work into, basically only to get pull requests merged into the “official” repo and then using that. It’s simply a matter of exposing some configuration parameters that the CVI gateway will need, that aren’t yet covered by the existing module.

Store and gateway

I’ve made a more-or-less blank repo for the RI Store. It’ll use the Coinbase and Paypal APIs for payment processing on the external side, and an internally-facing REST/JSON API that the CVI Gateway will use to validate user logins, and apply bandwidth limitation/QoS according to the user’s price plan. Early versions will be “unlimited data only” for the sake of technical simplicity, and that payment option will always be a supported service.

General puppetization

All my new site products are being made to deploy with Puppet, and being tested via a Debian Stable VM. That includes the CVI gateway and the Store. Over time, I’m going to retrofit existing repositories like this website and its mediagoblin component to deploy with Puppet, so I can setup a test version of the site in a VM in 20 minutes.

This isn’t just important for testing. In the event of an emergency, it’s really important to be able to salvage data from a failed system and set up a new one quickly. Regularly creating blank VMs and installing a complete copy of this site means that I’ll be constantly motivated to get the kinks out of the install process, and highly confident in my ability to deal with disaster. Ultimately, I also need to invest in data redundancy, but I’m still small-time enough that no one will be bothered if I don’t.