Git and .pycs – not the best of friends

Coming from a C++ background, using Python for the first time was pretty amazing.  The ease with which you can link in other libraries and quickly write code is fantastic.  Learning and using Git for the first time on the other hand – that took some adjustment.

Git and I are on fairly good terms these days, and I’ve learned to love some of the features it offers compared to TFS (the last source control system I was using), especially when used in conjunction with Github.  However I hit a problem the other day that made _zero_ sense to me – and took a surprisingly long time to track down, due to lots of false positives in figuring out the issue.

I’d been switching back and forth between a few branches, and all of a sudden my unit tests were failing, and the app appeared to be testing things that shouldn’t exist in this branch. Not only that, the app suddenly refused to run – I kept getting a pyramid.exceptions.ConfigurationExecutionError error.  I found that if I added in a bunch of routes that existed in the _other_ branch, suddenly everything worked.

Experienced Python/Pyramid coders will probably know exactly what the problem was, but I was stumped.  It turns out that our Git is configured to ignore .pyc files – which is normally what you want and works fine. However when files exist in one branch and not in another, it can cause all sorts of issues.

There are two main ways of solving this:

  1. Create a clean_pycs alias in your .bashrc file, that you can run to clean your pycs when you need to – i.e. alias clean_pycs='find . -name "*.pyc" -exec rm {} \;'
  2. Use a Git hook to automatically clean up the files every time you change branches – more info here.

There are other ways you can do it, mentioned in the follow-up to the blog post above, but these are the two easiest.  Which one you use is up to you – I’ve spoken to a few other coders at work, and because we work on a lot of different projects they tend to use the former, rather than having to add the hook in so many Git configs.

Hopefully this blog post helps somebody – I was incredibly thankful when I finally found the blog post linked above!

Learning Python

About a year ago my wife and I packed up all our things and moved to the Silicon Valley to work for SurveyMonkey.  As well as changing job, city and country, I switched from being a Windows application developer to being a backend web developer.  I spent over 8 years writing high performance trading applications in C++ and SQL under Windows, and traded it all in for Python, Javascript, open source libraries, virtual Linux environments, and OS X.  I did get to keep using SQL Server 2008 though surprisingly!

I have noticed in Silicon Valley people tend to be a lot more collaborative than in Melbourne.  I suspect it may be because of the heavier use of open source software, and also the rise of distributed source control like Git.  Back in Melbourne most technology companies seemed to use either Java, C++, or C# – when I left Python and Ruby were becoming more prominent, particularly due to a burgeoning startup community, but still pretty relatively rare compared to here.  These languages don’t seem to lend themselves as well to open source collaboration – however I will admit my experience was extremely filtered as a developer working in a Windows dev-shop that rarely used a third party library.

I’ve found the many tech blogs about Python and Pyramid extremely helpful when trying to solve problems, and in the spirit of this collaborative effort, would like to give back a little with what I’ve found.  I also find that writing up technology discussions and solutions very helpful for refining and consolidating my own thoughts, so even if this isn’t widely read, it’s a very useful exercise.

To kick off, I thought I’d mention something that seems to have caught surprisingly few people out on the web, but had me completely stumped a few weeks back at work.  It’ll follow in my next post.  Thanks for reading!