[LRUG] Why does test invocation affect result?

[Andrew Stewart]

Hello LRUG,

I have a Rails 3.0 app which generates PDFs.  There are a couple of different themes which change how the PDF content looks, e.g. fonts, layout, etc.

To test this I generated a reference PDF in each theme (via the first method below).  The tests generate the PDF again for each theme and verify the generated PDF and the respective reference PDF are identical (using FileUtils.identical?(a, b) ).

The weird thing is one theme's test pass or fail depending on how you run the test.  Specifically the first two ways pass and the third fails:

$ bundle exec ruby -Ilib:test test/unit/my_test.rb
$ bundle exec rake test:units TEST=test/unit/my_test.rb
$ bundle exec rake test:units

The theme's reference PDF is 399582 bytes.  The last way produces a visually identical PDF of 399583 bytes.  When I diff the PDFs I get:

260c260
< /BaseFont /AAAAAD+CaviarDreams
---
> /BaseFont /AAAAAA+CaviarDreams
548c548
< /BaseFont /AAAAAE+CaviarDreams-Bold
---
> /BaseFont /AAAAAB+CaviarDreams-Bold
1235c1235
< << /Length 5410
---
> << /Length 5409
1240,1257c1240,1257
[binary stuff snipped]

So my first question is why am I getting different results?  And another question that comes to mind: is there a better way to test all this?

Thanks in advance,
Andy Stewart
----
http://airbladesoftware.com