A/B Testing Instant.Page With Netlify and Speedcurve

This post ended up leading to the discovery of a bug in the way Chrome handles prefetched resources. I’ve written a follow-up post about it, and how it impacted the results of this test.

I’ve wanted to give instant.page a proper try out on my site for a while now. It’s a pretty clever approach. The script watches for when someone hovers over a link and then, when they do, uses the prefetch resource hint to get started fetching that page before the click event even occurs. I like the idea, in theory, quite a bit. I also like the implementation. The script is tiny (1.1kb with Brotli) and not overly aggressive by default—you can tell it to prefetch all visible links in the viewport, but that’s not the default behavior.

I wanted to know the actual impact, though. How well would the approach work when put out there in the real-world?

It was a great excuse for running a split test. By serving one version of the site with instant.page in place to some traffic, and a site without it to another, I could compare the performance of them both over the same timespan and see how it shakes out.

It turns out, between Netlify and SpeedCurve, this didn’t require much work.

Netlify supports branch-bases split testing, so first up was implementing instant.page on a separate branch. I downloaded the latest version so that I could self-host (there’s no reason to incur the separate connection cost) and dropped it into my page on a separate branch (very creatively called “instant-page”) and pushed to GitHub.

With the separate branch ready, I was able set up a split test in Netlify by selecting the master branch and the instant-page branch, and allocating the traffic that should go to each. I went with 50% each because I’m boring.

I still needed a way to distinguish between sessions with instant.page and sessions without it. That’s where SpeedCurve’s addData method comes into play. With it, I can add a custom data variable (again, creatively called “instantpage”) that either equals “yes” if you were on the version with instant.page or “no” if you weren’t.

<script>LUX.addData('instantpage', 'yes');</script>

I could have added the snippet to both branches, but it felt a bit sloppy to update my master branch to track the lack of something that only existed in a different branch. Once again, Netlify has a nice answer for that.

Netlify has a feature called Snippet Injection that lets you inject a snippet of code either just before the closing body tag or just before the closing head tag. Their snippet injection feature supports Liquid templating and also exposes any environmental variables, including one that indicates which branch you happen to be on. During the build process, that snippet (and any associated Liquid syntax) gets generated and added to the resulting code.

This let me check the branch being built and inject the appropriate addData without having to touch either branch’s source:

% if BRANCH == "instant-page" %
<script>LUX.addData('instantpage', 'yes');</script>
% else %
<script>LUX.addData('instantpage', 'no');</script>
% endif %

Then, in SpeedCurve, I had to setup the new data variable (using type “Other”) so that I could filter my performance data based on its value.

All that was left was to see if the split testing was actually working. It would have only taken moments in SpeedCurve to see live traffic come through, but I’m an impatient person.

Netlify sets a cookie for split tests (nf_ab) to ensure that all sessions that land on a version of the test stay with that version as long as that cookie persists. The cookie is a random floating point between 0 and 1. Since I have a 50% split, that means that a value between 0.0 and 0.5 is going to result in one version, and a value between 0.5 and 1.0 is going to get the other.

I loaded the page, checked to see if instant.page was loading—it wasn’t which meant I was on the master branch. Then I toggled the cookie’s value in Chrome’s Dev Tools (under the Application Panel > Cookies) and reloaded. Sure enough, there was instant.page—the split test was working.

And that was it. Without spending much time at all, I was able to get a split test up and running so I could see the impact instant.page was having.

It’s early, so the results aren’t exactly conclusive. It looks like at the median most metrics have been improving a little. At the 95th percentile, a few have gotten a hair slower.

It’s not enough yet to really make a concrete decision—the test hasn’t been running very long at all so there hasn’t been much time to iron out anomalies and all that.

It’s also worth noting that even if the results do look good, just because it does or doesn’t make an impact on my site doesn’t mean it won’t have a different impact elsewhere. My site has a short session length, typically, and very lightweight pages: putting this on a larger commercial site would inevitably yield much different results. That’s one of the reasons why it’s so critical to test potential improvements as you roll them out so you can gauge the impact in your own situations.

There are other potential adjustments I could make to try to squeeze a bit more of a boost out of the approach—instant.page provides several options to fine-tune when exactly the next page gets prefetched and I’m pretty keen to play around with those. What gets me excited, though, is knowing how quickly I could get those experiments set up and start collecting data.

find the cost of your paper

Sep 13, Grand Remembrances

Today is Grandparents Day in the United States. Being a Grand is a special honor. I feel very blessed that my wife and I have two grandchildren. We were able to visit them today. Yes, we are still being cautious with the coronavirus, but we also find it very difficult to not see them when they live so close. So today we did drop by to visit Jacob (age 10) and Sophia (age 7) along with their parents. We brought donuts and caught up with them. Our grandchildren are still pretty young and this is a precious time in their lives – and ours!

I wish I had known my grandparents better. We never lived in the same place. Dad was a career Air Force pilot, so we moved around a lot. But we did get to see them once in a while when they would visit us, or we them.

A Plague of Giants

There are five known magical ‘kennings’ or types: air, water, fire, earth, and plants. Each nation specializes in of these kennings, and the magic influences the society. There’s a big pitfall with this diversity of ability and locale–not everyone gets along.

Enter the Hathrim giants, or ‘lavaborn’ whose kenning is fire. Where they live the trees that fuel their fire are long gone, but the giants are definitely not welcome anywhere else. They’re big, they’re violent, and they’re ruthless. When a volcano erupts and they are forced to evacuate, they take the opportunity to relocate. They don’t care that it’s in a place where they aren’t wanted.

I first read Kevin Hearne’s Iron Druid books and loved them (also the quirky The Tales of Pell), so was curious about this new venture, starting with A PLAGUE OF GIANTS. Think Avatar: The Last Airbender meets Jim Butcher’s Codex Alera series. Elemental magic, a variety of races, different lands. And it’s all thrown at you from page one.

But this story is told a little differently. It starts at the end of the war, after a difficult victory, and a bard with earth kenning uses his magic to re-tell the story of the war to a city of refugees. And it’s this movement back and forth in time and between key players in this war that we get a singularly grand view of the war as a whole. Hearne uses this method to great effect.

There are so many interesting characters in this book that I can’t cover them all here. Often in books like this such a large cast of ‘main’ character can make the storytelling suffer, especially since they don’t have a lot of interaction with each other for the first 3/4 of the book–but it doesn’t suffer, thankfully. And the characterization is good enough, despite these short bursts, that by the end we understand these people and care about what happens to them.

If there were a main character it would be Dervan, a historian who is assigned to record (also spy on?) the bard’s stories. He finds himself caught up in machinations he feels unfit to survive. Fintan is the bard from another country, who at first is rather mysterious and his true personality is hidden by the stories he tells; it takes a while to understand him. Gorin Mogen is the leader of the Hathrim giants who decide to find a new land to settle. He’s hard to like, but as far as villains go, you understand his motivations and he can be even a little convincing. There’s Abhi, the son of hunters, who decides hunting isn’t the life for him–and unexpectedly finds himself on a quest for the sixth kenning. And Gondel Vedd, a scholar of linguistics who finds himself tasked with finding a way to communicate with a race of giants never seen before (definitely not Hathrim) and stumbles onto a mystery no one could have guessed: there may be a seventh kenning.

There are other characters, but what makes them all interesting is that they’re regular people (well, maybe not Gorin Mogen or the viceroy–he’s a piece of work) who become heroes in their own little ways, whether it’s the teenage girl who isn’t afraid to share vital information, to the scholars who suddenly find how crucial their minds are to the survival of a nation, to the humble public servants who find bravery when they need it most. This is a story of loss, love, redemption, courage, unity, and overcoming despair to not give up. All very human experiences by simple people who do extraordinary things.

Hearne’s worldbuilding is engaging. He doesn’t bottle feed you, at first it feels like drinking from a hydrant, but then you settle in and pick up things along the way. Then he shows you stuff with a punch to the gut. This is no fluffy world with simple magic without price. All the magic has a price, and more often than not it leads you straight to death’s door. For most people just the seeking of the magic will kill you. I particularly enjoyed the scenes with Ahbi and his discovery of the sixth kenning and everything associated with it. But giants? I mean, really? It isn’t bad enough fighting people who can control fire that you have to add that they’re twice the size of normal people? For Hearne if it’s war, the stakes are pretty high, and it gets ugly.

The benefit of the storytelling style is that the book, despite its length, moves along steadily (Hearne is no novice, here). The bits of story lead you along without annoying cliffhangers (mostly), and I never got bored with the switch between characters. It was easy to move between them, and they were recognizable enough that I got lost or confused. The end of the novel felt a little abrupt, but I guess that has more to do with I was ready for the story to continue, despite the exiting climax.

If you’re looking for epic fantasy with fun storytelling and clever worldbuilding, check out A PLAGUE OF GIANTS.

The post A Plague of Giants appeared first on Elitist Book Reviews.

The Artwork Of Gary Choo

Gary Choo is a concept artist/illustrator based in Singapore. I’ve know Gary for a good many years ( 17, actually ), working together in animation studios in Singapore like Silicon Illusions and Lucasfilm. Gary currently runs an art team at Mighty Bear Games, but when time allows he also draws covers for Marvel comics, and they’re amazing –

The Art Of Gary Choo
The Art Of Gary Choo
The Art Of Gary Choo
The Art Of Gary Choo
The Art Of Gary Choo

To see more of Gary’s work or to engage him for freelance work, head down to his ArtStation.

The post The Art Of Gary Choo appeared first on Halcyon Realms – Art Book Reviews – Anime, Manga, Film, Photography.

27