If you already have a computer, it's easy to start recording.  All you need is an adapter to plug in your guitar to your computer.  At Radio Shack it's only about $2.

Just plug in your guitar cord into the adapter and plug it to your sound card's line input or mic input.  Then open up your sound recorder program.  

Okay, the quality is not great.  And you don't get multi-track recording with your sound recorder program, but you should be able to see how easy it can be. Of course if you have a recent Mac and it came with iLife then you've got it made because it comes with GarageBand and you're all set for multi-track recording.

After you get your taste of recording to your computer, you'll want to go and get some kind of sound card / audio interface.