Tag Archives: tech

Galaxy Zoo iPhone/iPad app

General, , , ,

A few weeks ago I finally released my iOS app for Galaxy Zoo on the Apple App Store. Now that it’s been used a bit and I’ve had the chance to fix a couple of crashes, I’m announcing it properly. I am a little afraid of harsh reviews but please try it out and  be kind.

I’m still not quite as comfortable with it as I am with my Android app for Galaxy Zoo, partly because I just don’t feel at home with Objective-C and because iOS lacks something like SyncProvider. But it seems to work.

I actually finished writing the app about eight months ago, but I had to wait until Apple gave me permission to use the old “Galaxy Zoo” app name, allowing this app to appear as just a new version of the old (withdrawn) Galaxy Zoo app from 2014 or so. Joe Zuntz, who wrote the old app, was very helpful with that, pestering Apple until it happened. Unlike that old app, it should be possible to keep this one working even if the server changes, because it’s open source on github.

Here are some screenshots:

screenshot_iphone_4p7inch_classify
screenshot_iphone_4p7inch_list
screenshot_iphone_4p7inch_help

libxml++ 3.0 soon

General, ,

Kjell Ahlstedt has done some work on a new parallel-installable version of the libxml++ API, to fix some mostly-minor but annoying things that needed ABI breaks. You can see the code in libxml++’s git master.

He’s ready to release a stable .0 version soon. So now is the time to mention anything about the API that needs to be changed, please. People too often only look at these things right after the first stable release, when it’s too late.

C++ tuple utils

General, , ,

I’ve recently been been writing lots of modern C++ code with variadic templates. For instance , I’ve been trying to make libsigc++ use variadic templates instead of being a mess of generated code.

I often find myself needing utility functions and type traits to manipulate tuples, but the C++ standard library still only offers std::tuple_cat(). Writing these is awkward and that often stops me from experimenting quickly.

So I’m gradually gathering this code together in a little murrayc-tuple-utils library. I’d gladly change the name if this gets any use. Really, I’m surprised that nothing like this seems to exist already, apart from as part of larger projects such as boost::hana and boost::fusion. But boost is a really awkward dependency and those are larger projects with much grander goals.

So far murrayc-tuple-utils has:

  • tuple_cdr(): Removes the first element.
  • tuple_start<N>(): Takes the first N elements.
  • tuple_end<N>(): Takes the last N elements.
  • tuple_subset<pos, len>: Takes len elements, starting at pos.
  • tuple_interlace<T1, T2>: Takes elements from each tuple, interlacing (or zipping) them together.

For each of them, there are also type traits, such as tuple_type_cdr<>::type, though these are not so necessary now that C++14 has decltype(auto) for return types.

These are just enough code to make things work enough for me when I’m in a rush. I’m sure they can be improved, and maybe this is how to get those patches and pull requests.

The project has a complete autotools build structure, with “make check” tests, Doxygen documentation building, a pkg-config .pc file, etc, so you can try it out, improve it, and add to it, without having to mess around with that stuff.

Playing with xdg-app for PrefixSuffix and Glom

General, , , , , , ,

xdg-app lets us package applications and their dependencies together for Linux, so a user can just download the application and run it without either the developer or the user worrying about whether the correct versions of the dependencies are on the system. The various “runtimes”, such as the GNOME runtime, get you most of the way there, so you might not need to package many extra dependencies.

I put a lot of work into developing Glom, but I could never get it in front of enough non-technical users. Although it was eventually packaged for the main Linux distros, such as Ubuntu and Fedora, those packages were almost always broken or horribly outdated. I’d eagerly fix bugs reported by users, only to wait 2 years for the fix to get into a Linux distro package.

At this point, I probably couldn’t find the time to work more on Glom even if these problems went away. However, I really want something like xdg-app to succeed so the least I could do is try it out. It was pleasantly straightforward and worked very well for me. Alexander Larsson was patient and clear whenever I needed help.

xdg-app- builder

I first tried creating an xdg-app package for PrefixSuffix, because it’s a very simple app, but one that still needs dependencies that are not in the regular xdg-app GNOME runtime, such as gtkmm, glibmm and libsigc++.

I used xdg-app-builder, which reads a JSON manifest file, which lists your application and its dependencies, telling xdg-app-builder where to get the source tarballs and how to build them. Wisely, it assumes that each dependency can be built with the standard configure/make steps, but it also has support for CMake and lets you add in dummy configure and Makefile files. xdg-app-builder’s documentation is here, though I really wish the built HTML was online so I could link to it instead.

Here is the the manifest.json file for PrefixSuffix. I can run xdg-app-builder with that manifest, like so:

xdg-app-builder --require-changes ../prefixsuffix-xdgapp manifest.json

xdg-app-builder then builds each dependency in the order of its appearance in the manifest file, installing the files in a prefix in that prefixsuffix-xdgapp folder.

You also need to specify contexts in the “finish-args” though they aren’t explicitly called contexts in the manifest file. For instance, you can give your app access to the network subsystem or the host filesystem subsystem.

Creating the manifest.json file feels a lot like creating a build.gradle file for Android apps, where we would also list the base SDK version needed, along with each version of each dependency, and what permissions the app needs (though permissions are partly requested at runtime now in Android).

Here is the far larger xdg-app-builder manifest file for Glom, which I worked on after I had PrefixSuffix working. I had to provide many more build options for the dependencies and cleanup many more installed files that I didn’t need for Glom. For instance, it builds both PostgreSQL and MySQL, as well as avahi, evince, libgda, gtksourceview, goocanvas, and various *mm C++ wrappers. I could have just installed everything but that would have made the package much larger and it doesn’t generally seem safe to install lots of unnecessary binaries and files that I wouldn’t be using. I do wish that JSON allowed comments so I could explain why I’ve used various options.

You can test the app out like so:

$ xdg-app build ../prefixsuffix-xdgapp prefixsuffix
... Use the app ...
$ exit

Or you can start a shell in the xdg-app environment and then run the app, maybe via a debugger:

$ xdg-app build ../prefixsuffix-xdgapp bash
$ prefixsuffix
... Use the app ...
$ exit

Creating or updating an xdg-app repository

xdg-app can install files from online repositories. You can put your built app into a repository like so:

$ xdg-app build-export --gpg-sign="murrayc@murrayc.com" /repos/prefixsuffix ../prefixsuffix-xdgapp
$ xdg-app repo-update /repos/prefixsuffix

You can then copy that directory to a website, so it is available via http(s). You’ll want to make your GPG public key available too, so that xdg-app can check that the packages were really signed by you.

Installing with xdg-app

I uploaded the resulting xdg-app repository for PrefixSuffix to the website, so you should be able to install it like so:

$ wget https://murraycu.github.io/prefixsuffix/keys/prefixsuffix.gpg
$ xdg-app add-remote --user --gpg-import=prefixsuffix.gpg prefixsuffix https://murraycu.github.io/prefixsuffix/repo/
$ xdg-app install-app --user prefixsuffix io.github.murraycu.PrefixSuffix

I imagine that there will be a user interface for this in the future.

Then you can then run it like so, though it will also be available via your desktop menus like a regular application.

$ xdg-app run io.github.murraycu.PrefixSuffix

Here are similar instructions for installing my xdg-app Glom package.

I won’t promise to keep these packages updated, but I probably will if there is demand, and I’ll try to keep up to date on developments with xdg-app.

Galaxy Zoo app for iPhone: Learning iOS Development

General, , , ,

My Android app for Galaxy Zoo is doing pretty well. Creating it gave me a chance to learn Android development quite thoroughly. So I thought I’d do the same for iOS, with Objective-C. The Galaxy Zoo iPhone app is now mostly done, though it’s only available via TestFlight for now. Please send me your Apple ID (usually your email address) if you’d like to try it.

Over the years, I didn’t much believe the reports of Apple’s APIs, tools, and documentation being so wonderful. But I had hoped things would be better.  Android’s API and documentation can be frustrating, but developing for iOS feels worse, with rather primitive tools. It’s still doable, but I can’t think of anything about it that I prefer to Android development.

This is a rough collection of rants, so I reserve the right to revise my opinion when I have gained more complete understanding.

Objective-C

Learning Objective-C was not too difficult, though I still find its syntax and terminology arbitrarily different and often obstructive. Watching all of Stanford’s “Developing iOS 7 Apps for iPhone and iPad” course (on iTunes here too) helped me gradually get used to Objective C, XCode, and Apple’s *Kit APIs.

I chose to start with Objective-C rather than Swift, just so I’d have a chance to learn it and to avoid any problems caused by Swift still being fairly new. I plan to convert my code to Swift soon as its gradually becoming the default language for iOS development. Unfortunately it looks like Swift inherits most of Objective-C’s weird features.

Whereas in Java or C++, you would call a method on an instance like this:

SomeResult *someresult = something.somemethod(arg1, arg2);

in Objective-C, you’d send a message (not call a method) to an instance, like this:

SomeResult *someresult = [something somemethod:arg1 witharg2:arg2];

And if something is nil (null) then, unlike C++ or Java, there will be no runtime crash or exception – you’ll just get a null result. So you don’t need to check for nulls unless you really need the method to be called, though this leads to all kinds of extra implicit paths through your code. I secretly like this, but it is of course madness.

Users of C++ or Java will also be shocked at how it’s normal in Objective-C to add API to already-defined classes instead of providing the API via derived classes or helper methods. For instance, when using the (not from Apple) AFNetworking library, you can #import “UIImageView+AFNetworking.h” to add methods (messages) such as setImageWithURLRequest to the regular UIImageView class.

XCode

XCode is overly simple when it comes to basic code editing and debugging. I was amazed that I couldn’t just click on an exception’s backtrace to go to the relevant line of code. And I really missed the refactoring features in Android Studio or Eclipse.

XCode is not intuitive, forcing you to learn where to click and when, instead of learning how some structured code works. For instance, you need to know about the magic Ctrl-drag to access UI controls in your code, to respond to UI events from those UI controls.  Embedding your views into tab bars and navigation views is similarly obscure.

Auto Layout

Nothing has frustrated me more than Auto Layout in iOS. Amazingly, iOS had no real automatic layout until iOS 6 in 2012, and I don’t think it was really usable until iOS 7 in 2013. And developers still needed separate layouts (Storyboards) for iPhone and iPad versions of their apps until iOS 8 in 2014.

Auto layout seems to be a matter of playing with:

  • Constraints: You can add constraints, for instance to say that a child view (widget, control) should be the same width as another view , or that it should have a certain size of margin between itself and the leading (left) edge of its container. The XCode UI makes it really hard to see what constraints exist and the XML is obfuscated, so this is really difficult. You still need to specify positions and width/height for all your views, even if they would be changed by the auto layout, so you inevitably have to keep dealing with XCode’s warnings about the “frame” not being the same as it would be at runtime.
  • Content Hugging (vertical or horizontal): A high value stops the view from being made larger, instead “hugging” its content (such as text). I dislike this terminology particularly.
  • Compression Resistance (vertical or horizontal): A high value stops the view from being made smaller.

Content hugging and compression resistance are numeric values, so your layout tends to become littered with arbitrary values such as 100, 250, or 1000, as you try to make things work, with very little sense of overview of the whole layout.

I much prefer the automatic layout systems used in GTK+ and Android, where it’s mostly just a matter of using parent layout containers and telling them how to arrange their children. Android’s layout system is often cryptic, compared to GTK+’s simplicity, but the layout XML is readable so it’s fairly easy to try things out. Stackoverflow answers about iOS autolayout are all about where you should click in various versions of XCode, but the answers about Android layouts can just show XML samples.

Android child fragments can be difficult, but I eventually worked things out. In iOS/XCode still don’t have a satisfactory way to auto layout child containers that contain child views.

CoreData

CoreData seems to be a way to access a SQL-like database using generated classes. For instance. I find the manual code generation quite awkward, but I suppose it works. Most of my difficulties were to do with using CoreData with RestKit. But I eventually found its saveToPersistentStore() method to actually store the data reliably.

I’ve also missed having an equivalent for Android’s SyncAdapter, though I have not yet found the same UI performance problems that made it so necessary on Android.

Beta testing

I can understand Apple’s insistence on reviewing all app uploads to their app store. But it’s annoying that every version of private beta tests have to be approved too.And

To let someone beta test your app via TestFlight, you need to know their Apple ID and add it manually in the iTunes Connect website. In comparison, with Google’s Open Beta Testing for Android, you can just publish a URL that lets people opt in themselves.

And as far as I can tell, users don’t get any notification when you release a new beta version. So your old testers generally don’t even install the new version until you’ve bugged them individually to do so.

Coursera/Stanford course: Algorithms: Design and Analysis, Part 2

General, , , ,

A few weeks ago I mentioned completing Part 1 of the online Coursera/Stanford “Algorithms: Design and Analysis” course. Part 2 of Algorithms: Design and Analysis isn’t due to start again until next year, but I didn’t want to wait, so I enrolled in the archived version of the course to watch the videos and do the assignments. I should be ready to just reuse my work when Part 2 starts again for real.

Part 2 was where things got really interesting. The assignments required implementing these algorithms, though the course covered others too:

  • A Greedy Algorithm for job scheduling.
  • Prim’s and Kruskal’s minimum spanning tree algorithms. (Both O(m log n) but Prims does better on dense graphs, with more edges.)
  • Modifying a minimum spanning tree to identify clusters.
  • The Knapsack problem (Dynamic Programming – both bottom-up and recursive).
  • Shortest Path with the Belmann-Ford SSSP (Single-Source Shortest Path) algorithm (O(mn) and works with negative paths, but fails with negative cycles) as an alternative to Dijkstra’s Shortest Path algorithm (O(m log n) and works only with positive paths).
  • All Pairs Shortest Path with the Floyd Warshall algorithm (dynamic programming) (O(n3) and works with negative paths, though it fails when it detects negative cycles). Best for dense graphs.
  • All Pairs Shortest path with Johnson’s algorithm, via one call to Belmann-Ford on a modified graph and repeated calls to Dijkstra on a reweighted graph. (O(mn log n) and works with negative paths, but fails with negative cycles.) Best for sparse graphs.
  • A dynamic programming algorithm for the Traveling Salesman Problem.
  • Local search with the 2Sat problem, using Papadimitriou’s Algorithm.

I particularly enjoyed exploring “dynamic programming”, which is really just avoiding unnecessary repeated work after you’ve identified the appropriate sub-problems. It’s identifying the sub problems that is really hard. I enjoyed playing with bottom-up dynamic programming (filling in an array as you go, often discarding the n-2th set of results as you go), and top-down dynamic programming, also known as memoization (usually recursing into sub problems and ideally not doing as many sub problems as you’d do working bottom up).

While implementing a dynamic programming solution for the Traveling Salesman Problem, I learned about Gosper’s Hack for iterating over subsets. It’s now a personal favorite in my toolbox.

As with part 1 of the course, I am not allowed to publish the code of my homework solutions. But I did create a public version of the knapsack problem for solving the Make Change problem without a canonical currency (not a real world set of coins), using dynamic programming, though you shouldn’t use that as a first way to understand the classic knapsack problem. I also implemented a simple greedy algorithm for the Make Change problem with a canonical currency (real world set of coins).

The Make Change problem was interesting because I’ve read that people can and should learn to recognize NP-Complete problems, such as the traveling salesman problem. However, it is not obvious which sets of coins would cause the Make Change problem to be solvable with a greedy algorithm and which would need dynamic programming, though it might at first seem like a minor detail. (I haven’t actually read that paper yet.)

I’ve also been reading through Steven Skiena’s The Algorithm Design Manual book, which I can highly recommend. It’s more practical and enjoyable than the classic Introduction to Algorithms book by Cormen et al.

Update: I did part 2 for real when it started again. Here is my certificate:

 

Coursera/Stanford course: Algorithms: Design and Analysis , Part 1

General, , ,

screenshot_github_algorithmsAlthough I’ve been developing software for years, I noticed recently that I lacked the basic computer science knowledge that other people got at university, though it’s never been an issue outside of job interviews. I knew the basics of Big-O notation and how to use data structures but couldn’t describe exactly how various sort algorithms worked or how to analyze an algorithm’s performance from pseudo-code.

But this is all standard stuff now, so filling in the gaps in my knowledge seemed like a solvable problem. Over the last few weeks, I’ve worked through Coursera’s “Algorithms: Design and Analsis, Part 1” online course, provided by Stanford University. I was surprised to find myself enjoying it. It’s nice to get some insight into commonly used algorithms and data structures, and I guess it does help to inform choices made at a higher level, even if that’s only occasionally useful.

Mostly I enjoyed the programming exercises, writing implementations of Mergesort, Quicksort, Karger’s Minimum Cut Algorithm, Strongly Connected Components, Dijkstra’s Shortest-Path Algorithm, a 2-Sum Algorithm, and a Median Maintenance algorithm. Likewise, each week had a theoretical test, checking knowledge of stuff such as the Master Method, sorting algorithms, graph algorithms, heaps, (balanced) binary trees, hash tables, and bloom filters.

Hopefully I’ll keep it in all my head from now on, but that is always easier when you’ve written actual code that you can look back at. I used C++, but you can use any programming language, and nobody checks your code. Of course, I’m not allowed to publish my homework solutions.

I had already been slowly reading through Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, which is hard going. It also has exercises, but I was far more motivated to complete the Coursera exercises, whose aim was always to get the correct specific numerical answer, so you knew when you had the code working properly. After I finished the course, I found it easy to then read the relevant chapters, which offer much more in-depth analysis than the Coursera course.

Although the participants are all still waiting for their certificates (presumably some skeumorphic image file), I’m sure that I’ve passed with 80-something percent. I’d have done better but I started the course after some homework deadlines had already passed. The final test also showed that I need to be more familiar with logarithm equivalences and geometric progressions. I do have Maths and Further-Maths A-Levels, but it’s been a long time.

Unfortunately, part 2 isn’t due to start again until some time in 2016. But I think I can do the course in the meantime, just without earning an official score.

Update: Here is my certificate for part 1:

murrayc-2015-09-24_coursera_stanford_algorithms_part1

Update: I did part 2 too.