Mac OS

Eclipse – Hanging on Android SDK Content Loader or Error Markers

icon-huge

Me and Eclipse (especially on Mac) do not get along. We have a lot of issues, and one of the big ones is the constant hanging when I have to re-open Eclipse to switch projects or branches.

This is probably a post for future me, when I’ve forgotten how to do it and need to Google it again, but maybe it’ll help someone else out.

So, do you open Eclipse and constantly get the IDE hanging on either

  • Android SDK Content Loader
  • Android SDK Resolving error markers

If you do, I’ve found the following resolves it every time (it might also solve a couple of other hangs, but I’ve not been able to test that)

  • Force Quit Eclipse (it probably won’t shut down properly if it’s hung)
  • Open Terminal on your Eclipse workspace
  • Move the .projects folder so we can use it again later
    • mv .metadata/.plugins/org.eclipse.core.resources/.projects projects
  • Reopen Eclipse, your projects should now load
  • Once loaded, close Eclipse again (it’ll probably complain about being unable to save the workspace, that’s fine)
  • Move the projects folder back
    • mv projects .metadata/.plugins/org.eclipse.core.resources/.projects
  • Reopen Eclipse and find you can actually do some work!

Objective-C Swizzling without Side Effects

<tl;dr>

Using method_exchangeImplementations results in a number of unexpected and dangerous side effects which in most cases you’ll want to avoid. You should use method_setImplementation and swizzle a C-style function instead to give yourself more control over your applications behaviour.

</tl;dr>

 

Swizzling is an incredibly powerful, and incredibly dangerous, feature of Objective-C that in the wrong hands can cause a serious amount of problems. A lot of this stems from a lack of understanding about what exactly swizzling is, and how it works. It’s also incredibly common to swizzle using a method that can cause quite serious side effects that if not tidied up can cause quirky and seriously hard to track down bugs.

I initially came across a few of these issues when unit testing our payment pipeline on iOS as I discussed previously. Swizzling worked like a charm, but required a bit of rooting around in the Apple docs to figure out why I was getting the behaviour I was.

Anyway, this post will cover a few of those problems I’ve just described

  1. What exactly is an Objective-C method
  2. Use method_exchangeImplementations at your peril
  3. Prefer method_setImplementation to swizzle without generating any side effects

 

What is an Objective-C method

Objective-C is built on C and as a result it’s methods are actually defined as C-style structs (typedef’d as Method)

struct objc_method
{
     SEL method_name;
     char* method_types;
     IMP method_imp;
}
typedef struct objc_method *Method;

The method_name is the name of the selector (as called by [self selector_name]), the method_types is an encoding of the return value and parameters passed to the function, but the most important entry for now is the method_imp.

This is how it’s defined in objc.h

typedef id (*IMP)(id, SEL, ...);

This means that every Obj-C method is actually just a C-style function that passes through the object, the command and the function parameters. So, if the method called uses any properties or other selectors (calling self. or [self selector_name]), it uses the id object passed through to the function which is where your problems can start to arise.

It’s this structure that’s key to understanding swizzling and the effects it can have.

 

The Consequence of method_exchangeImplementations

The most regular way of swizzling functions is through the use of method_exchangeImplementations. This does exactly as it says, swaps the two method implementations allowing calls to the original method to call the swizzled method instead.

@implementation my_interface

     -(int)swizzle_returns_22
     {
          return 22;
     }

     -(void)display_swizzle_results
     {
          // I’m using SKPayment* as it relates well to the next unit testing post
          SKPayment* payment = [[SKPayment alloc] init];
          NSLog(@“Default call: %d", (int)payment .quantity);
          
          // Get our methods to swap
          Method m1 = class_getInstanceMethod([payment class], @selector(quantity));
          Method m2 = class_getInstanceMethod([self class], @selector(swizzle_returns_22));

          // Exchange them
          method_exchangeImplementations(m1, m2);

          // Call quantity again
          NSLog(@“Replaced call: %d", (int)payment.quantity);
     }
@end

The output of display_swizzle_results is

Default call: 1
Replaced call: 22

But what if display_swizzle_results is modified to include an additional call to the replacement method?

-(void)display_swizzle_results
{
     // I’m using SKPayment* as it relates well to the next unit testing post
     SKPayment* payment = [[SKPayment alloc] init];
     NSLog(@“Default call: %d", (int)payment .quantity);
     
     // Get our methods to swap
     Method m1 = class_getInstanceMethod([payment class], @selector(quantity));
     Method m2 = class_getInstanceMethod([self class], @selector(swizzle_returns_22));
     
     // Exchange them
     method_exchangeImplementations(m1, m2);
     
     // Call quantity again
     NSLog(@“Replaced call: %d", (int)payment.quantity);
     
     // What is the result of this call?
     NSLog(@“Local method: %d", [self swizzle_returns_22]);
}

This new output is

Default call: 1
Replaced call: 22
Local method: 0

0 is probably not what you expected

  • If you didn’t see the 2 lines prior, you’d expect 22
  • If you did see the prior 2 lines, you’d expect 1

So we have two problem here, the first being the fact that you cannot possibly know who’ll use or modify your code in the future, and you’ve altered your code so any call to your functions actually calls something completely different.

The second is the call actually returns garbage.

The first problem is one of documentation and clarification. You could document the heck out of your definition and hope someone in the future bothers to read it, but it’s something a couple of breakpoints and a bit of wasted time would eventually figure out – “oh, it’s not actually calling that function”.

The second would be much harder to track down – “whats it calling and what the hell is 0?”.

So why is the call return 0?

We don’t have the source code to [SKPayment quantity] so we can’t look at the call directly, but that chances are that internally it’s using ‘self’ to access either another selector or another property. Since our implementations are called passing through ‘self’ those properties or selectors that the compiler thinks exists, actually don’t. It’s expecting an object or type SKPayment and instead it’s getting an object of type my_interface.

Who knows if what the internal call to self is calling something that exists in your object or doesn’t, or what the possible side effects will be.

None of this behaviour is ever desired, and should be avoided at all times.

 

Using method_setImplementation Instead

Using method_setImplementation avoids the side effects as it doesn’t swap the functions around (resulting in the call to your client function still calling your client function) and instead it simply replaces your target.

But if you look at the syntax of method_setImplementation it no longer takes a selector (it doesn’t need a new selector so why should it) so switching over isn’t as trivial as just changing the function name.

IMP method_setImplementation(Method method, IMP imp)

An IMP is nothing more than a function pointer that is explicitly passed the two ‘secret’ parameters.

typedef id (*IMP)(id, SEL, ...);

So, in our case, we simply need to define a C-style function to perform the behaviour of our original selector

int swizzle_returns_22_c_style(id self, SEL _cmd)
{
     return 22;
}

Now, to avoid any side effects or unexpected behaviour, here’s the original function again

-(void)display_swizzle_results
{
     // I’m using SKPayment* as it relates well to the next unit testing post
     SKPayment* payment = [[SKPayment alloc] init];
     NSLog(@“Default call: %d", (int)payment .quantity);
     
     // Set our new function
     Method originalMethod = class_getInstanceMethod([payment class], quantity);
     method_setImplementation(originalMethod, (IMP)swizzle_returns_22_c_style);
     
     // Call quantity again
     NSLog(@“Replaced call: %d", (int)payment.quantity);
     
     // What is the result of this call?
     NSLog(@“Local method: %d", [self swizzle_returns_22]);
}

This new output is

Default call: 1
Replaced call: 22
Local method: 22

The behaviour of these two functions is now significantly easier to understand, has zero side effects, and doesn’t result in us overriding behaviour we have no control or understanding of.

So, the next time you need to swizzle any default behaviour you otherwise wouldn’t have any control over, make sure you look at the available API and certainly default to preferring method_setImplementation over method_exchangeImplementations.

Injecting data into Obj-C readonly properties

<tl;dr>

If you want to inject data into an object that only has read-only properties, swizzle the synthesised getter function so you can inject the data you need at the point it’s accessed.

For example

// Interface we want to override
@interface SKPaymentTransaction : NSObject
     @property(nonatomic, readonly) NSData *transactionReceipt;
@end

//
// Returns an invalid receipt
//
NSData* swizzled_transactionReceipt(id self, SEL _cmd)
{
     return @“my receipt”;
}

//
// Test our receipt
//
- (void)test_InvalidTransactionSentViaAPI_VerificationFails
{
     // Create an invalid transaction
     SKPaymentTransaction* invalidTransaction = [[SKPaymentTransaction alloc] init];

     // Replace transactionReceipt with our own
     Method originalMethod = class_getInstanceMethod([invalidTransaction class], @selector(transactionReceipt));
     method_setImplementation(originalMethod, (IMP)swizzled_transactionReceipt);
}

</tl;dr>

 

I recently needed to set up some client side unit tests for our iOS receipt verification server. This server takes a payment object and verifies it with Apple to check if it’s actually a legal receipt and if it is, the content is awarded to the player. Server side, this is pretty simple, but it’s an important server step and should anything happen its possible for people to be locked out from getting their content.

So it’s important we have tests that send legal, invalid and corrupt data to the server and we need to test through the client API otherwise it’s not a test that can be 100% reliable.

Our verify API looks something like the following

+(BOOL) queueVerificationRequest:(NSString*)productId withTransaction:(SKPaymentTransaction*) transaction;

It takes an SKPaymentTransaction because thats the object the client deals with. We’re going to use internal data such as NSStrings or NSData, but it shouldn’t be the clients responsibility to query the SKPaymentTransaction to get the relevant information. Should the structure change, our API also breaks and that’s not acceptable.

So we’re stuck with the SKPaymentTransaction and its API looks something like this

@interface SKPaymentTransaction : NSObject
     @property(nonatomic, readonly) NSError *error;
     @property(nonatomic, readonly) SKPaymentTransaction *originalTransaction;
     @property(nonatomic, readonly) SKPayment *payment;
     @property(nonatomic, readonly) NSArray *downloads;
     @property(nonatomic, readonly) NSDate *transactionDate;
     @property(nonatomic, readonly) NSString *transactionIdentifier;
     @property(nonatomic, readonly) NSData *transactionReceipt;
     @property(nonatomic, readonly) SKPaymentTransactionState transactionState;
@end

Now in our case, we’re interested in using [SKPaymentTransaction transactionIdentifier] and [SKPaymentTransaction transactionReceipt], which both need to be managed inside our verification call so we’ll need SKPaymentTransaction objects that return different values depending on what we’re testing.

And since they’re readonly, we can’t just set them manually.

Initially, I tried to use class extensions, just to add the behaviour I was looking for, as you’ll often add internal extensions to present readonly properties externally but support readwrite properties internally.

@interface SKPaymentTransaction()
     @property(nonatomic, readwrite) NSString *transactionIdentifier;
     @property(nonatomic, readwrite) NSData *transactionReceipt;
@end

Compiles fine but at runtime it generates an unknown selector error. This is because the read only property has already be synthesised, and while the compiler now thinks it can see a setter, at runtime it’s not present.

So, my second attempt was to derive from SKPaymentTransaction and add the functionality there, passing through the base SKPaymentTransaction but using the derived type to set the value.

@interface SKPaymentTransactionDerived : SKPaymentTransaction
     @property(nonatomic, readwrite) NSString *transactionIdentifier;
     @property(nonatomic, readwrite) NSData *transactionReceipt;
@end

Fortunately, the compilers a bit smarter this time and warns me before I even start

error: auto property synthesis will not synthesize property 'transactionIdentifier' because it is 'readwrite' but it will be synthesized 'readonly' via another property [-Werror,-Wobjc-property-synthesis]

At this point I was a bit stuck until @pmjordan suggested I swizzled the transaction identifier and receipts to return the values I’m interested in testing, rather than setting the values directly.

But what needs to be swizzled when we’re attempting to override an Obj-C property defined as follows

@interface SKPaymentTransaction : NSObject
     @property(nonatomic, readonly) NSData *transactionReceipt;
@end

Properties are automatically synthesised (unless explicitly defined) so we’re actually looking at the following selectors

@interface SKPaymentTransaction : NSObject
     // Selector used to return the data
     -(NSData*)  transactionReceipt;

     // This would be defined if we’d specified the property as readwrite
     // -(void)     setTransactionReceipt:(NSData*)receipt;
@end

So at this point, our tests look like the following

//
// Returns an unique invalid receipt
//
NSData* replaced_getTransactionReceipt_Invalid(id self, SEL _cmd)
{
     static int runningId = 0;
     ++runningId;

     NSString* receiptString = [NSString stringWithFormat:@"replaced_getTransactionReceipt_Invalid %d", runningId];
     return [receiptString dataUsingEncoding:NSUTF8StringEncoding];
}

// Checks we fail through the API
- (void)test_InvalidTransactionSentViaAPI_VerificationFails
{
     //
     // Set up ...
     // 

     // Create an invalid transaction
     SKPaymentTransaction* invalidTransaction = [[SKPaymentTransaction alloc] init];
     
     // Inject our invalid receipt method instead of using the default [SKPaymentTransaction transactionReceipt]
     Method originalMethod = class_getInstanceMethod([invalidTransaction class], @selector(transactionReceipt));
     method_setImplementation(originalMethod, (IMP)replaced_getTransactionReceipt_Invalid);

     // Test our verification
     [HLSReceipts queueVerificationRequest:@"test_InvalidTransactionSentViaAPI_VerificationFails" withTransaction:invalidTransaction];

     //
     // Test validation ...
     //
}

As a result of this call, when the library calls [SKPaymentTransaction transactionReceipt] it will instead call replaced_getTransactionReceipt_Invalid which gives us the ability to pass through any receipt we want, including real, invalid and corrupt ones as our tests dictate.

It’s worth noting here that I’m using method_setImplementation rather than the usual method_exchangeImplementations which I’ll explain in a following post soon.

GLFW – Compiling Source and Creating an Xcode Project

GLFW is a great library – easy to use, great documentation covering it’s API design. But if you’re not on Windows where they provide pre-built libraries, going from a source download to a compiling Xcode project is _not_easy.

Since brew (as of writing) only provides GLFW 2, you need to build GLFW 3 yourself.

There is barely any information on it at all, it assumes you know exactly what you’ve downloaded (the source and some CMake files btw) and that you’ll be able to figure it all out (there is some additional information if you root around the GitHub forums but it’s not exactly easy to find).

So I’m posting up how I got GLFW to compile and running in Xcode here in-case anyone else starts banging their head against a brick wall.

First off, download the source followed by Cmake (I downloaded the 64/32-bit .dmg for 2.8.11.2).

Now to get it to compile you need to set up a few options. You can do this in the Cmake GUI, but I found it much easier to do it manually. Browse to GLFW/Src and open config.h.in.

This is a settings file for the compile you’re about to perform. There are a lot of settings in here you can play around with (they are explained in the README.md file in the root GLFW folder), but I enabled the following

  • _GLFW_COCOA
  • _GLFW_NSGL
  • _GLFW_NO_DLOAD_WINMM (this is a Windows only define, but I enabled it so adding it here anyway)
  • _GLFW_USE_OPENGL

 

Save the file and then you’re ready to compile.

Open the Terminal and browse to the root of the GLFW folder and enter the following commands

cmake .
make install

The Cmake command will set up the project ready to be built. It’ll take the config file we modified earlier and create a valid config.h file and it’ll carry out any other set up needed as well.  Calling make builds the project, the install command installs the library in it’s default location.

Now you’re ready to add GLFW to your Xcode project. To keep it simple I’ll step through adding it to a new project, once you can do that adding it to an existing one is pretty easy.

  • Open Xcode and create a new OSX Command Line Tool project
  • In the project settings add the path to the GLFW include file in the ‘Header Search Paths’ and the path to the libglfw3.a lib to the ‘Library Search Paths’ option (both of these paths are shown in the install make output)
  • Add libglfw3.a to the Link build phase
  • You also need to add the following Frameworks to the build phase to allow the project to compile
    • IOKit
    • Cocoa
    • OpenGL

 

You can now copy/paste the sample code provided on the GLFW website into the existing main.c file, hit compile and you have a running GLFW project.

Note that the sample code doesn’t call glclear before glfwSwapBuffers so what you get in the open window is undefined.

Ruby, Jenkins and Mac OS


I’ve been using Jenkins as my CI server for a while and though user permissions has always been a bit of an issue (I’ll explain why in another blog post covering my Unity build process) running command line tools has never been to much of a problem once it got going.

At least not until I tried to run Ruby 1.9.3 via RVM on our Mac Jenkins server.

I’d developed the Ruby scripts outside Jenkins so I knew they worked, but when I came to run them through Jenkins (using nothing more than the ‘Execute Shell’ build step) it started behaving strangely. Running the script caused the step to fail instantly claiming it couldn’t find any of my installed Gems.

A quick ‘gem query –local’ on the command line listed all the gems I needed were there.

As an experiment I added a build step that installed the Trollop gem (a fantastic gem, you should check it out!) to see if that made any difference, in effect forcing the install in the service that would run the Ruby script. I was surprised when the install worked, but it installed it for Ruby 1.8 rather than Ruby 1.9.

Adding ‘ruby –version’ as a build step showed that for some reason the Jenkins server was using Ruby 1.8.7 rather than 1.9.3.

It turns out that RVM is not available when run through a non-interactive shell script, which is a bit inconvenient when you need it run as part of an automated process.

Searching around I came across this Stack Overflow answer suggesting I make changes to my .bash_profile but those changes were already present meaning I had little luck in finding a solution.

Other suggestions involved rather convoluted steps to just get the thing working, something I neither had the time for nor thought should be required.

Luckily Jenkins provides a RVM Plugin which allows the build step to run inside an RVM managed environment meaning it will respect the RVM settings I’ve set outside of Jenkins…

Now when running ‘rvm list’ via a build script shows that we have multiple versions of Ruby available with 1.9.3 set as the default.

And all of a sudden my Ruby scripts work like a charm!

Bug or Bad UI: Find and Replace in XCode 4

As applications get larger and more complicated with every iteration it sometimes becomes difficult to know if incorrect behaviour is down to a bug or badly designed user interface. XCode 4 is an application that has some quirky design choices, some which I really like and some which just baffle.

Here’s one a I found recently when trying something as simple as ‘Find and Replace’

Go to the search tab and search for something (preferably something that exists!). So in the following screenshot I’m searching for ‘defgroup’ which is a Doxygen documentation tab.

Now switch over to ‘Find’ and, as you would do if you did this a while later, change what you last searched for to what you want to replace. In this example, I’m now looking to change ‘client’ to ‘user’. Looking at the following screenshot, what would you expect to be replaced?

In my mind, I expected ‘client’ to be replaced with ‘user’. After all, that’s what I’ve put next to the Magnifying Glass icon – a universal icon for ‘search’ – and the results are for the last search I did, not this one.

But if I select Preview…

It’s actually replacing the results of the original search for ‘defgroup’ and not what I actually asked for, which was to replace ‘client’ with ‘user’. If I’d gone and selected Replace All then the same thing would have happened.

To get it to replace ‘client’ rather than ‘defgroup’ I need to either

  • Go back to Find search for ‘client’ then switch back to Replace and do it again
  • When entering the new search term, hit enter after typing ‘client’. This won’t do the replace, it’ll redo the search.

 

So, is this a bug or a failure of UI design? Should Preview and Replace All take what I’ve just put in the search box (in which case it’s a bug) or should it use what I last searched for (in which case it’s a UI issue).

Personally, I see this as a bug. As I work down the panel, I add what I what to replace with the new term then select Replace All. Since we read from left to right and top to bottom, information at the top of the screen is more important (the new term) than at the bottom (the results of the last search).

But then, some people could argue it’s a UI issue, and in that case an improvement to this work-flow from a UI perspective would cause this bug to never appear in the first place.

Macintosh Hackintosh

I recently bought a Mac. I wanted a small, easy to use media centre to sit next to my TV and while I originally looked into getting a Dell Zino I thought it’d be a perfect time to get an unintrusive Mac and headed off to the Apple Store and picked up a Mac Mini.

So far I’m loving it (even after having to return the first Mini due to a faulty graphics card and then the wireless keyboard due to a faulty space bar!) and as an introduction to Mac’s it’s been worth it. I don’t really do anything ‘Maccy’ on it other than use XBMC, browse the internet and use Aperture but I’m actually enjoying the experience rather than wanting to throw the machine out of the window (having said that, I’ll still dual boot to Windows should I want to do any hobby development with XNA).

Since I was so impressed with it, I thought it’d be worth trying to install Mac OS X on my little Dell Mini 9. I love my Mini 9 but even with a fresh install it’s been struggling even running Windows XP (taking 10 minutes to boot up(!), constantly freezing and generally being pretty naff). Installing OS X, I hoped, might give it a bit of a kick up the backside.

I also network the two machines, using the Dell as a little media server and Time Machine seemed to meet my server back up needs and it’s free out of the box. What’s not to like?

I was worried that this would involve kernel hacking, boot strap loaders, luck and quite a bit of swearing but surprisingly it was stupidly easy… Following the Hackintosh instructions for the Dell Mini 10 resulted in a Mac powered Mini 9 in about 90 minutes.

But then things got a little bit complicated. Anyone keeping track of the Hackintosh ‘scene’ will know that OS X 10.6.2 dropped support for Atom processors, put it back in, then dropped it in the final release, which would mean no OS X on my Dell Mini. But not updating an OS isn’t really that good an idea.

A few quick hits on Google brought up a possible hack. Changing various kernel files, kernel panics (seriously I’d panic trying this) and any other number of things that can go wrong (I’m not worried about anything breaking because it’s not going to be the end of the world. I just don’t really have the time to try and figure out what I need to do and do it, rather than it just working out of the box).

But there really are some cool people on the internet. Using NetBook Installer (a sister project to the Netbook BootMaker tool which was used to install OS X in the first place).

Updating from OS X 10.6.0 to 10.6.2 was as simple as

(I probably didn’t need to run Netbook Installer the first time but it only take 5 minutes).

And literally that was is. A smooth copy of OS X 10.6.2 running on my compact Dell Mini 9 as a quiet, low power media server.

It’s all good.

(It might be noticed that I’ve not been blogging much recently. I’ve not run out of things to say it’s just that with moving house, Christmas and lots (and lots) of DIY, the time has just not been there. Service will resume at some point – soon)