In the course of working on my next demo for my demo-a-week project, I'm doing a lot of procedural texture generation. The ability to quickly test modifications to my algorithms is crucial, so I decided to whip up a simple texture editor, Straytex (named after Straylight, my demogroup).

It likely only works on the latest Chrome dev channel build (edit: also works on Safari and Firefox, but I'm not sure which versions) and is ugly as sin (if anyone wants to help out with either of these issues, it'd be greatly appreciated!), but it gets the job done. At the moment, the interface is pretty simple: you can select the texture size, change the random seed, modify the source, and tag your changes. The tag feature is especially handy -- you often accidentally stumble onto cool effects, so the ability to tag these to go back to them later is insanely useful.

You can see it live here, and check the source out on Github. I hope you enjoy it.

Happy Hacking,
- Cody Brocious (Daeken)

For a long time, I've sat on the sidelines of the demoscene, generally watching in awe as ASD, Farbrausch, Conspiracy, and others produce amazing works of art. I recently decided that I'd jump in and see what I can do, and to accomplish that I've decided to start releasing a demo a week. My goal is to get my feet wet and learn how to actually build something beautiful.

Today I'm releasing my very first real demo ever, Waveride, which I decided to hack together over the course of the last few hours. There's not much to it -- a wave plane consisting of spheres and droplets falling onto it with locations based on the music data -- but I don't think it's bad for a first shot. I'm planning on improving it over the coming weeks, before moving on to something new. At the moment, it's Windows-only, but pure OpenGL -- it should work fine in Wine. I'm going to port it to OS X and Linux before moving on.

Anyway, here's a link if you want to check it out in its current incarnation: . I've already received a good bit of information on what I need to fix (the nauseating rotation is absolutely first up on the list, I swear!), but please give me anything you can think of. This is an experiment for growth, and I can't grow if I don't know what I need to improve upon. The NFO is below.

Happy Hacking,
- Cody Brocious (Daeken)

Waveride by Straylight
                - March 25, 2010

Welp, not much to say.  I said I was going to create a demo a week until I had
a clue what I was doing.  Waveride is the first of these demos, and I'm pretty 
damn proud of it.  It's not perfect, and I'm not terribly happy with having 
used an off-the-shelf song for it, but eh.

Decided to sit down and write a demo, and I did it.  That was 1am, this is 8am.
Can't complain too much.

Developed and designed by Daeken.
Music is Miriel by Nightbeat (I believe this is cool -- if not, please, please 
let me know, and I'll change it at once.)

Greets to:
Nightbeat
ASD
SVatG
Lateralus
Kewlers

Until next time,
- Daeken

(Edit: The Daeken Discount Program is closed.  While an interesting exercise, it resulted in only one sale, but it was of the 25% discount level so I can't complain too heavily.  If nothing else, it was fun to try something.)

A long while back, I saw people selling shares in themselves. That is, you invest in the person and get a portion of their earnings and such, just like investing in a company. While I like the idea in theory, I can't see selling off a large (1 %) part of myself that I'll likely never get back. However, the idea's stuck with me. So while I was laying in bed last night, I came up with an idea.

The Daeken Discount Program

I'm launching an experiment. As of today, you can buy a lifetime discount on any of my products; rules and restrictions are below.

• $50 gets you a 5% discount
• $90 gets you a 10% discount
• $200 gets you a 25% discount

I see this as a win-win. If no one buys into it, I'm out an hour worth of work and write this off as an interesting failure. If people do buy into it, I potentially lose some future profits, but the people who would do so are going to be more likely to buy from me anyway.

Why should anyone buy into this?

I think that people will see that I build cool things, and it's a small enough amount of money that everyone that buys into it will eventually at least break even.

How does it work?

The discount will be tied to the email you put in the Paypal purchase (although if you end up changing emails, I'll move it over to another, of course), and when you use it for purchases on anything of mine, you'll immediately receive the discount.

What does it apply to?

Any product purchase of mine, whether one-time or recurring. The obvious exception is if I don't have the right to do this, e.g. a product I build for someone else. However, all of my future companies will fall under this.

How long will you run this?

Probably no more so than a month or so. If it succeeds, I'd rather not have a million people with lifetime discounts on my products; if it fails, there's no point in leaving it up.

What have you built and what are you building?

Product-wise, I haven't built much; the biggest thing I've built was Alky, which allowed the conversion of Win32 binaries to run natively on OS X and Linux. It was a marginal success, although it ended up failing later for business reasons.

These days, I'm working on a few things:

• Renraku: This will eventually be combined with
• OpenBAMF/IREctive: Reverse-engineering platform and module store (My primary for-profit project right now)
• Books
  • The Emulator's Handbook: A book on building an emulator from start to finish. There's simply nothing there yet, which is a damn shame -- we need to get people involved here.
  • So far unnamed: A book on reverse-engineering game protocols and emulating them.

In the future, I'd like to be developing and selling Eyetaps and other hardware around Renraku.

What if you never build anything interesting? / What if you never start another company?

In the (I hope, unlikely) event of one of these happening, I'm deeply sorry. I do my best to create things that people will want, but it's entirely possible that I'll fail. If I end up working a day-job for the rest of my life and someone buys into this program, I'll do whatever is in my power to give you the discount, even if I have to pay the difference myself.

Can I buy it multiple times?

No, sorry, but this does apply on top of any other discounts available.

What if you renege on the deal?

Then I'd hope that I'd be outted as a fraud and prevented from ever doing business again. Seriously, I wouldn't do business with someone who pulled something like that, and I'd sincerely hope that others wouldn't either.

Questions?

Post a comment or email me at [email protected]. I'm curious to see whether or not this takes off; would be cool for it to do so.

Happy Hacking,
- Cody Brocious (Daeken)

Earlier this week, I had to deal with some files in Python's marshal format (some .pyc files, specifically) in Ruby and discovered that the details of this format aren't documented. Since it's meant to be purely internal, the Python team has decided not to document it in any way.

The marshal format is used in .pyc files, lots of internal storage for random apps, etc. It's a shame that it's undocumented, as this means that there are, to my knowledge, no implementations for other languages. This also means that if you have a malicious marshal blob, you have to load it up with Python to play around with it; not a good idea.

Fortunately, you can read the source (Python/marshal.c) and figure out how it works pretty easily. However, to make it even easier, I've decided to write up some simple documentation on the format. I'll give types as int/uint where n is the number of bits. The object type indicates that this is a marshalled object.

Format

The marshal format in and of itself is very simple. It consists of a series of nested objects, represented by a type (uint8 -- a char, in fact) followed by some serialized data. All data is little-endian.

Note: I wrote all of this for the 2.x line. I don't know how much has changed in 3.x.

Types

Constants

These types contain no data and are simply representations of Python constants.

• 0 (TYPE_NULL) -- Used to null terminate dictionaries and to represent the serialization of a null object internally (not sure if this can happen or not).
• N (TYPE_NONE) -- Represents the None object.
• F (TYPE_FALSE) -- Represents the False object.
• T (TYPE_TRUE) -- Represents the True object.
• S (TYPE_STOPITER) -- Represents the StopIteration exception object.
• . (TYPE_ELLIPSIS) -- Represents the Ellipsis object.

Numbers

• i (TYPE_INT) -- Represents a int on a 32-bit machine. Stored as an int32.
• I (TYPE_INT64) -- Represents a int on a 64-bit machine. Stored as an int64. When read on a 32-bit machine, this may automatically become a long (if it's above 2**31).
• f (TYPE_FLOAT) -- Represents a float in the old (< 1) marshal format. Stored as a string with a uint8 before it indicating the size.
• g (TYPE_BINARY_FLOAT) -- Represents a float in the new marshal format. Stored as a float64. (Thanks to Trevor Blackwell for noting that these are not float32 (along with TYPE_BINARY_COMPLEX).)
• x (TYPE_COMPLEX) -- Represents a complex in the old (< 1) marshal format. Contains the real and imaginary components stored like TYPE_FLOAT; that is, as strings.
• y (TYPE_BINARY_COMPLEX) -- Represents a complex in the new marshal format. Stored as two float64s representing the real and imaginary components.
• l (TYPE_LONG) -- Represents a long. Haven't yet figured out how this works; I'll update shortly with that.

Strings

• s (TYPE_STRING) -- Represents a str. Stored as a int32 representing the size, followed by that many bytes.
• t (TYPE_INTERNED) -- Represents a str. Identical to TYPE_STRING, with the exception that it's added to an "interned" list as well.
• R (TYPE_STRINGREF) -- Represents a str. Stored as a int32 reference into the interned list mentioned above. Note that this is zero-indexed.
• u (TYPE_UNICODE) -- Represents a unicode. Stored as a int32 representing the size, followed by that many bytes. This is always UTF-8.

Collections

• ( (TYPE_TUPLE) -- Represents a tuple. Stored as a int32 followed by that many objects, which are marshalled as well.
• [ (TYPE_LIST) -- Represents a list. Stored identically to TYPE_TUPLE.
• { (TYPE_DICT) -- Represents a dict. Stored as a series of marshalled key-value pairs. At the end of the dict, you'll have a "key" that consists of a TYPE_NULL; there's no value following it.
• > (TYPE_FROZENSET) -- Represents a frozenset. Stored identically to TYPE_TUPLE.

Code objects

Code objects (like that in a .pyc file, or in the func_code property of a function) use the c (TYPE_CODE) type flag. Even in the case of the top level (as in a .pyc), they represent a function.

They consist of the following fields:

• argcount (int32) -- Number of arguments.
• nlocals (int32) -- Number of local variables.
• stacksize (int32) -- Max stack depth used.
• flags (int32) -- Flags for the function.
  • 0x04 -- Has *args.
  • 0x08 -- Has **kwargs.
  • 0x20 -- Generator.
  • This list is not all encompassing; certain __future__ declarations will set their own flags.
• code (object) -- String representation of the bytecode.
• consts (object) -- Tuple of constants used.
• names (object) -- Tuple of names.
• varnames (object) -- Tuple of variable names (this includes arguments and locals).
• freevars (object) -- Tuple of "free" variables. (Can anyone clarify this a bit?)
• cellvars (object) -- Tuple of variables used in nested functions.
• filename (object) -- String containing the original filename this code object was generated from.
• name (object) -- Name of the function. If it's the top level code object in a .pyc, this will be <module>.
• firstlineno (int32) -- First line number of the code this code object was generated from.
• lnotab (object) -- String mapping bytecode offsets to line numbers. Haven't delved into the details here.

RMarshal

I've implemented support for unmarshalling objects as well as reading .pycs in a Ruby gem called RMarshal. You can get it from Gemcutter or from Github.

Closing

Hopefully this will be of use to someone. One potential use is in studying malicious marshalled data; the Python guys strongly recommend against unmarshalling untrusted data, but we all know how well such notices are regarded. In addition, it may help you manipulate Python bytecode from non-Python languages.

Drop me a line if you do anything cool with it.

Happy Hacking,
- Cody Brocious (Daeken)

Renraku has seen a renewed surge of energy in the past few weeks, and has made big strides. Here's what's going on.

Where we are

Renraku now runs in hosted mode on top of MS.NET/Mono. This enables a much, much quicker development/debugging cycle, and allows us to start working on the GUI while the low level details are still being worked out. There's still work to be done here, but it's a good start. We've also made the leap to Rake from NAnt; dealing with the bulky, inflexible build files just got to be too much. A few hours of hacking and Rake now fits our purposes perfectly.

After hosted mode came up, we made a push to get video/mouse services built on top of SDL, enabling us to start developing the GUI. You can see an initial GUI coming up and displaying our logo here:

In addition, Capsules (check out this blog post if you don't know what this is: Renraku OS: Networking, Hosted Mode, Moving Forward) are in progress and are taking shape; the implementation is still rapidly changing, but we seem to have a solid idea of what they're supposed to look like and how they interact with each other.

Where we're going

The project is now moving in parallel on two efforts: Userspace (GUI, applications, storage), and native (kernel, compiler, drivers). This will all converge over the next 6 months.

We're aiming to make two releases between now and July 4th, which will serve to get the code in the hands of developers leading up to v1.0. We now have a solid roadmap for Renraku v1.0, which we aim to release on July 4th, the one-year aniversary of the project. The roadmap is as follows:

Goal

The project goal is that v1.0 should go out on July 4, 2010. This marks the first anniversary of the project and will be the first major release.

What should it do?

You should be able to start Renraku in hosted or native mode (IA32 native only) and bring up a usable GUI. You should be able to browse files, start applications, edit configuration, and run UI tests. Depending on time constraints, we may or may not have additional applications (and a game?)

What needs to be done?

Compiler:

• Exceptions
• Generics
• Array bounds checking
• Emit proper class data for reflection

Gui:

• Basic toolkit for controls
• Image rendering
• Font rendering
• Vector support?

Graphical shell:

• Window management
• Desktop icons
• Some sort of system tray/menu

Kernel:

• Tracebacks on exceptions
• Garbage collection
• Memory management
• Storage
  • Low level storage service (hard drive access)
  • FAT32 filesystem service
• Networking
  • TCP
  • IP and UDP supporting fragmenting
  • More robust DHCP
  • DNS
  • Routing
  • Transparent object remoting
• Video driver on IA32 (better than VGA)

Services:

• Capsule implementation
• Service documentation

BCL:

• Migrate to Mono BCL
• Implement more of the BCL, if migrating to the Mono BCL isn't possible/practical

Applications:

• GUI console
• File browser
• Basic text editor
• Image viewer
• Some basic GUI game?

Build System:

• Allow portions of the code to be tagged as platform-specific, rather than the large file lists in the Rakefile
• Integrate building a LiveCD with a complete Renraku system.
• Build Renraku installer (XXX: May get pushed back to v2)

How you can get involved

At this point, we need a few things: Application developers, service developers, kernel developers, and compiler developers. Want to write the first Doom port to run on a managed OS, a Twitter client, a completely new web browser, or a sane networking stack? Here's your chance.

No matter how you want to help out, or if you just want to ask some questions, join us at #renraku on Freenode IRC.

Points of contact:

• Daeken (Cody Brocious, ) -- Kernel lead; Kernel, services.
• ircubic (Daniel Bruce, ) -- Userspace lead; GUI, applications, storage. Documentation lead. Build system.
• nrr (Nathaniel Reindl, ) -- BCL lead. Memory management lead.
• dublindan (Daniel Kersten, ) -- Capsule lead.

Happy Hacking,
- Cody Brocious (Daeken)