DevStick

From Iseborn Wiki
Jump to: navigation, search

Note: See DevStick/Installation for the easiest way to create a new DevStick.

DevStick is my (Boise's) project for keeping a complete development environment, as well as my ongoing projects (including the DevStick project itself), on a USB stick.

Why on Earth would I want that?

I use both my active computers (the stationary at home, and my laptop), as well as my PC at work, and sometimes a friends computer or even a public one, for development. So except for the fact that I think it is an interesting project in itself, there are two main reasons for having it all on one device:

  1. Software: If I have everything on a USB stick, I will never need to think about which software is installed on this or that computer. And even when I have the software installed on all computers, it can be a pain to keep updating them all to the same version (when I am even allowed to do that) - when all programs are installed on a USB memory there are no such problems.
  2. Projects: I have no working strategy for keeping track of all the projects I am working on. My memory sucks in general, and I usually have no idea at all where I keep all the files that I work on. So if I can keep it all in one place, even on an as easy-to-loose storage device as a USB stick, it is a WIN situation for me as long as I remember to take backup copies every now and then :-)

Real and potential problems with running software from a USB stick

Software installation

Installing software on a portable device is not usually as easy as it might sound. Almost all software that runs under MS Windows (and, yes, I really need to use that) can not simply be copied to some arbitrary folder and then just stays there without affecting its surroundings. No, any normal Windows application will update the Windows registry, install a bunch of DLLs and other data in the user'a folder, shared applications folders, and so on. All of this is on the local computer. Move the USB drive to another PC and it will simply not work.

Some developers actually think of this and creates special stand-alone versions for installation on (for example) a USB stick.

Other applications have a very small "footprint" on the installation PC. Small enough for someone on the web to discover what needs to be done, and then publish a step-by-step instruction for how to put it on a stick.

Then there are some projects out there with the sole purpose of making standalone versions of other developers software. The only one I remember the name of is PortableApps. I am not to fond of their setup and the "PortableApps.com Launcher", but I use a couple of their conversions separately, e.g. PortableGoogleChrome.

And finally, there are applications that no one sane would even consider putting on a USB stick, e.g. MS Office.

The point here is that you cannot just assume that you will be able to install any program on the stick, but here is Google a true friend. Check there first.

A USB device does not have a fixed drive letter

When you insert a USB stick in a PC, it will be assigned a drive letter (e.g. "E:"). The next time you insert the same stick in the same PC it may very well get a different drive letter, and when you put it in another computer it will probably get a different one. You can force Windows to always assign the same letter to a particular USB stick, but that does not help when you insert it into a computer where that letter is already taken by another device (especially if the PC is not yours to tamper with).

The problem with this is that the drive letter is a part of every directory path in MS Windows. Sometimes you cannot see the drive letter but it is still there. This is actually quite a big problem for our DevStick.

Say for example that your DevStick gets the drive letter D: one day. You start some application (from DevStick of course), creates a new file and saves it in some folder on the stick. The next day you sit at another PC, you start the same application from the stick and opens the same file from the "Recent files" menu. "File not found!"

What happened is of course that since your stick got another drive letter this day, the path to the file you saved the day before, and that the application stored in some "recent.xml" (or similar) is no longer valid.

This example is not a very serious problem. You soon understand what happened, and know where to find the file. But what if you need to do that with every file, all the time? And I can promise you (from own experience) that most application saves paths to stuff that you had no idea that they even used. Finding those erroneous drive letters and fixing the problem is a real pain.

Another common example of where this is a problem is the shortcuts in the DevStick root folder. The may point to a file only one folder level below them, but they still will not work when the drive letter changes. Well, actually most of them will work, since when you activate a shortcut, Windows will make an automatic look-up to search for the target file if path does not exist. But the shortcut icons will not work.

I solved this problem by writing the Perl script fix_paths.pl. This script will fix the drive letter in all paths that it can find in the setting files determined by a rule set in the script. It will also fix the drive letter in all the shortcut (.lnk) files in the DevStick root directory. You run this script by double-clicking the PrepStick.bat batch file in the root directory when you have inserted the stick in a computer.

Size and speed of the USB stick

Most of today's Windows applications are rather large, even the smaller ones (pun intended). So I would not recommend digging up some old 16 MB stick from the desk drawer to run DevStick on. Another very good reason not to do that is speed. Such a stick would probably have USB 1.0, which is way, way too slow.

So how much storage capacity do you need then? That is not an easy question to answer since it obviously depends on which programs you are going to need on it, and what you intend to use it for (for example, writing source code does not require much memory at all, while working with advanced graphics, or video requires a lot more space.

I am far from finished with installing everything I want/need on my stick, but right now (2013-05-26), I have used exactly 3 GB on my DevStick, out of which the Program folder uses 2.5 GB. So I should say that, for my own purposes, a 4 GB drive would be too small for comfort, but 8 GB would be more than enough.

When it comes to speed, the first thing to check is the USB version. Of course, a USB 3 stick does not actually have to be faster than a USB 2 stick. USB 3 just have the capacity to transfer data a lot faster than USB 2 :-)

I started the DevStick endeavor with an 8 GB Verbatim "Tough-'n'-Tiny" USB 2 stick, and that worked fine for me. But then I though, "If I'm going to sit at my really fast stationary PC (with SSD drives) at home, and run the software from the DevStick with crappy old USB 2, I'm just gonna go crazy". So I bought myself a 64 GB (!) Transcend JetFlash 780, USB 3 stick, which promises (up to) 210MB/s read and 140MB/s write speed. I have not verified the actual speed, but i can tell you that it is really fast. That is, when you have a free USB 3 port on the PC.

Now, I said above that 8 GB would be more than enough for me, so why 64 GB? The reason is simply that when I compared the different sizes that this stick came in, I found that the more storage capacity they had, that faster they where, and I wanted the fastest. So size (at least in the terms of storage capacity) really do seem to matter.

Tip: When on the subject of buying a USB stick for the DevStick, there is one more thing to consider. Check out the discussion about auto-run functionality for USB sticks in PrepStick.bat Usage. Maybe you want one that identifies itself as a CD-ROM drive instead of a USB drive...

DevStick-specific scripts and similar

PrepStick.bat

PrepStick.bat is a batch (.bat) file located in the DevStick root folder. Its purpose is to load Perl so that it then can run fix_paths.pl (see that script for more information).

PrepStick.bat Usage

The script opens a Windows Command window to run in and to print the log to. After execution is completed, this window will be closed automatically after five seconds (if everything went alright), or stay open until you close it (if there where errors or warnings).

You should make it a habit to always run PrepStick.bat (by double-clicking it in the DevStick root folder) every time you have inserted the DevStick in a computer. It only takes a couple of second to run, and it is better to make that effort rather than to discover later that you forgot to do it.

I tried to make the script auto-run when the stick is inserted (through an autorun.inf file in the root directory, but that did not work. I then found out that, starting with Windows Vista, Microsoft has disabled the auto-run function specifically for USB drives (for security reasons).

You could get around that by using a special USB stick that identifies itself as a CD-ROM instead of a USB drive, but personally I do not think it is worth it. Another way is to use third-party software, such as USB Safely Remove under Other software. With that you can make the DevStick auto-run on insertion, though that only works on computer where you have USB Safely Remove installed.

PrepStick.bat Maintenance

If you install something that is not part of the "standard" DevStick (see XXX below), or if you have any other user files or other that contain paths, you should update PrepStick.bat in the DevStick root folder so that it passes the names of all files there that (may) contain paths, as argument to the fix_paths.pl script. In other words, you should add the file names (separated by spaces) to the FILES environment variable.

fix_paths.pl

As described above, a USB drive does not necessarily get the same drive letter every time you use it, and this can lead to big problems for the DevStick development environment. So I wrote the fix_paths.pl Perl script to get around that problem.

This script (located in /Program/bin/) will scan selected folders and files on the DevStick, searching for paths and correcting the drive letter where needed.

There are two ways to determine which folder and files to scan:

  1. The script "knows" about common settings files for the "standard" applications on DevStick (see Maintenance below), and will correct those files automatically when run.
  2. You can provide a list of file names as arguments to the script, and it will then correct those (in addition to the known files). You should do this from the PrepStick.bat script (that is, if you need to use this method) rather than creating new scripts.

Updated files are only written if they have actually been changed by the script, and if an update is needed, the script will always rename the original file with the extension ".backup. The latter does not apply to modified shortcuts (below).

In addition to the files described above, fix_paths.pl will also correct the drive letters in all the shortcut (.lnk) files in the root folder of the DevStick. Shortcuts will generally work without this change since Windows will perform an automatic "look-up" of the target file if it is not found in the exact path stated by the shortcut, but that means that you may not be starting the version of the program that you wanted. And also, in that case, the shortcut icon will not work until you have used the shortcut.

Correcting the shortcuts is actually a bit more tricky since they are binary files, but it is still easy when using the Perl Win32::Shortcut CPAN module. For each shortcut, the script will check and correct (if necessary) the following attributes: Path, ShortPath, WorkingDirectory and IconLocation.

fix_paths.pl Usage

You should only run this script through PrepStick.bat. Not that it does not work otherwise (as long as you remember that you need to load Perl first). It is just better to make it a habit to use PrepStick.bat.

fix_paths.pl Maintenance

Whenever a new "standard" application is added to DevStick, this script will likely need to be updated so that it can find and correct the settings file(s) for the application.

WorkShell.bat

See DevStick/WorkShell.

SetEnv.bat

Installing software on the DevStick

You can of course install anything you want on your 'stick, that is not what this section is about. What I describe here is the software that I consider to be a part of the DevStick project and should be installed on every stick (not that you really need to).

Any program you install on the DevStick should be placed in the /Program folder. Each section below will briefly describe each piece of software to install, where to get it (it is all free stuff), and anything you need to know about how to get it to run off of the USB stick.

Note that I do not even try to list version numbers here. There would be no point to that. Just take the latest version you can find. Exceptions to this rule means that the version is in some way important, so read the information.

Code::Blocks - IDE for cross-platform C/C++ development

MinGW

MSYS

wxWidgets

Jansson - C library for working with JSON data

Download the source code from http://www.digip.org/jansson/. Choose the zip archive (e.g. for Jansson 2.4 here rather than downloading directly from the Git repository.

Also download the HTML documentation, since it is a bit tricky to build it locally.

Unpack the Jansson source directory under /Programs/ (e.g. /Programs/jansson-2.4/).

Open an MSYS shell window and go to the Jansson source directory you just unpacked.

Run the following commands (note that this may take some time):

./configure --prefix=/w/Program/jansson-2.4
make
make install

When that is done, you should test the result by running:

make check

All done.

Android SDK

I have chosen to use the ADT Bundle version of the Androide SDK. This version also includes Eclips, so the installation instructions will also include Eclipse on the DevStick.

  • Go to http://developer.android.com/sdk/index.html and download the Android ADT Bundle for Windows to your hard-drive. You should use the 32-bit version for compatibility with most PC:s.
  • Unpack the zip file in the /Programs directory of the DevStick.
  • Rename the newly unpacked /Programs/sdk directory as /Programs/android-sdk.
  • Now we need to tell Eclipse which JDK version to use, namely the one (or one of the versions if you have installed more than one) on the DevStick. Let us say we want to use /Programs/jdk6 since that it the default for DevStick:
Open the file eclipse/eclipse.ini and add the following two lines right before the existing "-vmargs" line:
-vm
../jdk6/bin/javaw.exe
  • Save the file.
  • Create a shortcut to eclipse/eclipse.exe in the root directory of the DevStick, and rename the shortcut as just "Eclipse".
  • Double-click the shortcut to start Eclipse.
  • The dialog "Workspace Launcher" will be displayed. Use the "Browse" button to create and select the directory /Dev/workspace on the DevStick as you workspace. Make sure to check the "Use this as the default and do not ask again" check-box, and then click OK.
  • You will then probably get some messages about installing and/or updating some stuff. Just follow the on-screen instructions and do that. Though, when all the necessary stuff is installed, you will be asked to close and re-open the Android SDK Manager. Do that, but when the SDK Manager reappears, remember that you do not have to install any of the stuff that is now available there, unless the SDK Manager manager specifically tells you that you should.

EasyPHP-DevServer

EasyPHP :

Eclipse

Since my main purpose of using Eclipse is for developing Android applications, I have chose to use the ADT Bundle version of the Android SDK. This bundle includes a version of Eclipse, so see the Android SDK section for how to (also) install Eclipse on the DevStick.

FileZilla

I have not yet found an FTP client software that I really like, but FileZilla is ok to use, free, very commonly used (even where I work), and relatively easy to install un a USB stick. Check out this article on their wiki to see how.

Google Chrome Portable

JDK

jEdit still has issues with Java 7, so I use the latest Java 6.

I recall seeing an instruction about installing JDK on a USB stick, but I just copied the C:\Program Files\Java\jdk_1.6.0_42 folder to /Program/jdk6 on my DevStick.

There is of course no reason not to also install Java 7 on the DevStick if you want to. Just make sure that jEdit is run with Java 6 (see jEdit.bat).

jEdit

jEdit is my favorite source code editor.

There is no standalone version for installation on a USB stick. I just copied the installation folder from my home PC to the USB drive and wrote a special start script, see jEdit.bat.

Strawberry Perl

Strawberry Perl

Portable Python

As the name "Portable Python" indicates, this is indeed a Python version that was created specifically for installment on (e.g.) USB drives, so there is really nothing to the installation.

The home page for the Portable Python project is [1]. Go there and click "Download" (or use this link), select the version you want, download it ant run it. Remember - this python version is made for installing on a USB drive, so the only thing you need to get right is the installation folder.

About selecting a version: I do not know a lot about Python, but there is a big difference between v2 and v3. They are basically just like v5 and v6 of Perl, i.e. the later version is completely re-written and not 100% compatiible with the "older" version. And "older" is quoted because both major versions are still alive and kicking, both for Python and Perl.

My point is that I keep using Perl 5 (latest version at time of writing is 5.16.2) since no one I know, and certainly not my workplace, is using Perl 6. And the same goes for Python - v2 suits me better since that is what "everyone else" is using.

This does not mean that the same goes for you, but I would say that if you do not know that you actually need to use Python v3, use v2 since more people use that, and it is not as if v2 is dead, it is updated in pretty much the same frequency as v3.

Other software (\Program\Installable software)

The DevStick folder \Program\Installable software contains a number of applications that cannot be installed on the DevStick, but that you may want to install on the computer you are using it from.

This software is described in the sections below.

USB Safely Remove

A really nice little program for handling USB devices connected to your PC.

Free trial for 30 days, then a small amount for lifetime use (one user, multiple computers) and one year of free updates.

Backup med Transcend Elite

I am not really familiar with backup software, so I cannot compare this with other similar products. It came with my lates USB stick and is pretty easy to use.

Tortoise SVN

You can survive with just the SVN plugin for jEdit, but that is really just an extension for integrating subversion with the project viewer. Meaning that the functionality is centered around working with the project viewer. And even if that is what you normally do (as it is for me), sometimes it is just not good enough.

Tortoise SVN is free-ware, very capable, and the interface is very easy to use (if you know how subversion works).

Source Code Pro

Source Code Pro is a font that is specifically designed for source code editing, and I like it a lot. It is free to download from Adobe hos GitHub.com.

A font needs to be installed on every computer that you want to use it on, but this is easy. Differs between Windows versions though, so you have to check it up.

I recommend using the TTF version (simply because that one works on all Windows versions).

You should install all the font styles, but at the very least you will need the "regular" and "bold" styles. If you only install "regular", Windows will simulate bold characters by widening the regular style. That means that bold characters are slightly wider than regular characters, which in an editor with syntax highlighting will look like crap on the screen.

DevStick Menu

I thought it would be enough just to keep shortcuts to all the installed programs in the root of the DevStick, but it has already become difficult to find what I am looking for there. So I decided that I need some sort of menu system where I can arrange the shortcuts.