Multiple Models, One Paginator

While working on YaBa, I found that I really needed a method to take items from multiple models, and paginate them as one major entity. In the case of YaBa I have galleries, articles, and the most common model of story/blog posts. Now, sure, there are likely a few different ways I could have gone about this. The easiest would have been to create a more general model, and used that for everything, however I wanted them to be separate. I wanted distinct articles and galleries that stood as their own entities when necessary, and didn’t get clutterred up in the main feed. I need articles to be able to be buttonized on the main navigation bar, and I wanted galleries to be easily sortable in their own story list. All that being said, at times I needed them all to be very similar in structure, and to be able to be listed together in searches, archives, and the main page feed.

The three main protagonists in this story thus had to be built with similar traits in mind. They all had to be taggable, have slugs, tie to categories, and be searchable without offending the others. This obviously was pretty straight forward, not a big deal. Honestly, would have been a great place to leverage some inheritance really, but hindsight’s 20/20 and apparently my foresight wasn’t. Not a big deal, wouldn’t have saved me all that many lines of code anyways. Besides, the problematic part was more in making these three different models merge into one entity, and still be usable in pagination scenarios (that ruled our itertools, which works great if you don’t need pagination). I was very lucky to stumble upon the MultiQuerySet Snippet at Django Snippets. This incredibly helpful little snippet allowed me to chain multiple querysets into one larger queryset. Now I had the ability to merge my 3 querysets chock full of data into 1 master queryset to render to my templates in a paginated format. There was still one minor problem, MultiQuerySet doesn’t do anything to help with the aspect of sorting.

By default, when using the MultiQuerySet snippet mentioned previously, all the items are sequentially thrown into the list, making your front page kind of strange. Visitors aren’t very likely to want to push through ALL of your stories, and then ALL of your articles, just to see the newest gallery. If they are willing to do that, you have the best visitors ever, or you’re running a self-help website for insomniacs with too much time on their hands. I had to find a crafty way to sort out the querysets into something more usable. Below is what I came up with:

def story_list(request):
stories = Story.objects.all().order_by('-created')
galleries = Gallery.objects.all().order_by('-created')
temp = MultiQuerySet(stories, galleries)
front_page = []
for x in temp:
front_page.append(x)

front_page.sort(key=sort_by_date, reverse=1)
paginator = Paginator(front_page, 5)
page = int(request.GET.get('page', '1'))
posts = paginator.page(page)

Currently on the front page, I don’t list articles, but I do list both galleries and your more general blog posts. So, walking through this, let’s start from the top. We grab all the stories and galleries, sorting them in the newest first, oldest at the back style. Then I create a MultiQuerySet objects in a temp variable that contains my two querysets. In return I get a nice queryset that I really can’t do much with sorting wise, since it’s not a “real” queryset, but it sort of is. The best way I found to handle the sorting action was to toss all the items into a list, and then use the built in Python list sorting functionality to sort it all out. As you can see I’m using a key function in the sort method of ‘front_page’. That function’s incredibly simple:

def sort_by_date(x):
return x.created

We’re sorting by each item’s created date asset that is in the list. After that we have a nicely sorted queryset like object, that for all intents and purposes at this point does exactly what we need it to do. We pass it off to the Paginator, let it do it’s thing, and then I render it off to the template for the story_list function. Now I know, it doesn’t look incredibly complex, and it actually is pretty easy. However, I hadn’t really found anyone else mentioning a method of doing this, so I figured I’d share with the world how I went about it. I’m actually really interested in how anyone/everyone else is tackling a similar problem though, since I don’t declare my method to be the best. It’s just a means to an end I suppose. :)