From CVS to Git to Gitorious!

Migrating from CVS to Git

Last week I’ve offered myself to migrate some ~300 repositories to git. Not an easy task at first, but with the right tools at hand the task becomes manageable. Installing cvs2git, and following its documentation will get you started. In Ubuntu that is as simples as:

sudo apt-get install cvs2svn

I know it’s weird, but cvs2git is bundled in cvs2svn… go figure.

But migrating hundreds of repositories isn’t a task to do manually, so I created a script for automating the process. As I had access to the server files, migrating was easier then I expected. My directory structure was something like:

  • cvs_project_1
    • repo_1
    • repo_2
    • repo_3
  • cvs_project_2

I’ve decided to migrate one project at a time, making it straightforward to verify each repo. My script is the following, bare in mind that it my have some flaws, it worked for me. Test it before erasing your old CVS data.

#!/bin/bash
# Copyright (C) Pedro Kiefer

for f in `cat repo_list`;
do
	FOP=${f/\//\-}
	echo "===== Creating git repository for ${f/\//\-/}/";
	sed -e "s/__REPO__/${f/\//\\/}/g" my-default.options > $FOP.options;
	cvs2git --options=$FOP.options
	rm $FOP.options
	mkdir $FOP.git
	cd $FOP.git
	git init --bare
	cat ../cvs2svn-tmp/git-blob.dat ../cvs2svn-tmp/git-dump.dat | git fast-import
	cd ..
done

The script takes a repo_list file with a list of paths to the CVS repositories. Creating this list is quite easy, something like this should work. Be sure to remove CVSROOT and the root directory.

find cvs_project_1/ -maxdepth 1 -type d | sort > repo_list
vim repo_list

The other file the script need is my-default.options, which is the configuration file used by cvs2git. Most of the default values are good, but you really want to add a list of cvs commiters – so you can map the cvs login to a name + email. The other change need is on the line that sets the repository path. For the script to work you need to have it set as __REPO__. Like this:

run_options.set_project(
    # The filesystem path to the part of the CVS repository (*not* a
    # CVS working copy) that should be converted.  This may be a
    # subdirectory (i.e., a module) within a larger CVS repository.
    r'__REPO__',

That’s it, just run the script, and voilà, git repositories for all your cvs modules.

From Git to Gitorious

The second part of my task was importing all of those git repositories to my local Gitorious install. Again, doing it manually is not the right way to do it. After asking about it on gitorious mailing list and learning some ruby, I’ve created this little script. It creates all the repositories for a given project. The projects were created manually on gitorious, as I had only 6 projects – extending the tool to support the creating of projects should be easy.

After using the script above, I had the following directory structure:

  • project_1/
    • repo_1.git
    • repo_2.git
    • repo_3.git

      The scripts takes as argument the project name, which should be equal to the one you created on gitorious web interface. The script scan the project directory and creates the matching gitorious repositories, copying the data to the newly created repository. Some magic regexp was added to remove version numbers and set uniform names to the new repositories. You might want to edit this to your taste.

      By the way, this is my very first ruby programming, don’t expect it to be pretty!

      #!/usr/bin/env ruby
      # encoding: utf-8
      #--
      # Copyright (C) Pedro Kiefer
      #
      # Mass migrate git repositories to gitorious
      #
      #++
      
      require "/path/to/gitorious/config/environment.rb"
      require "optparse"
      
      def new_repos(opts={})
        Repository.new({
          :name => "foo"
          }.merge(opts))
      end
      
      current_proj = ARGV[0]
      
      @project = Project.find_by_slug(current_proj)
      
      Dir.chdir(current_proj)
      puts Dir.pwd
      files = Dir.glob("*.git")
      
      files.each do |f|
        orig_repo = f
        f = f.gsub(/\.git$/, "")
        f = f.gsub(/_/,"-")
        
        # has version?
        version = f.match(/-([0-9](.[0-9][0-9]*)+)(-)?/)
        f = f.gsub(/-([0-9](.[0-9][0-9]*)+)(-)?/, "")
        
        desc = "Repository for package #{f.downcase}\n"
        desc << "Package version #{version&#91;1&#93;}\n" if version
        
        print "Creating repository for package #{f} ... " 
        
        @repo = new_repos(:name => f.downcase, :project => @project, :owner => @project.owner, :user => @project.user, :description => desc)
        @repo.save
        path = @repo.full_repository_path
        Repository.git_backend.create(path)
        Repository.create_git_repository(@repo.real_gitdir)
        @repo.ready = true
        @repo.save 
      
        FileUtils.cp_r(["#{orig_repo}/branches", "#{orig_repo}/info", "#{orig_repo}/objects", "#{orig_repo}/refs"], @repo.full_repository_path)
        puts "Ok!"
      end