Adventures in DIY Ad Servers

David: Thanks again for joining me today. I'm here speaking on behalf of Timehop and their programmatic journey for mobile in-app monetization. So this is a part two. However, for those of you who haven't heard part one, or who haven't heard my previous talks, don't you worry, I'm gonna go over some bullet points, some talking points, and hopefully fill in any gaps that you may have so you all are up to speed. So let's get technical.

Little bit about myself. Hi, my name is David Leviev. I joined Timehop a little under two years ago. And before my time, Timehop, I would say, was monetizing their inventory pretty standard for a mobile publisher. What I mean by that is that they sought out after a mediation platform and sold their SDK, sought out after various SSPs that were compatible with that mediation platform, integrated their SDKs, adjust their CPMs on the fly, deprioritized and prioritized their waterfall, cross their fingers and hope they did a good job. With that said, they had a whopping $2 overall eCPM and an abysmal 30% to 40% overall fill rate, so no numbers that any publisher wants to hear.

So I come in, coming from a previous SSP that was eventually acquired so I was kinda looking for a new venture, joined Timehop to see what I can do. So quickly, I was looking at different opportunities, looking at different answers, different SDKs, different mediation partners, plugging in, testing, and really trying to figure out who's the best solution out there. What I found was that, specifically for Timehop, what we were looking for just never existed. So what we decided to do was a little bit unconventional. We decided to build our own ad server.

So this is Nimbus. It's a true header bidding ad server focused and optimized for mobile via server-to-server. So what does that mean, right? So us, as a publisher, we were…we didn't want to, like, replicate what was out there in the market. We didn't want to do what other ad servers did, which was pretty much mirroring what was available on desktop and trying to kind of make it a mobile-friendly service. It just doesn't work like that. So we decided to kinda brainstorm, sit around the round table and kind of figure out how to approach this effectively.

The way we did it is to start…we decided to do things a little bit unconventional. So when we were speaking to different ad servers, we noticed that they all had this one documentation and they were just like, "Okay, this is the way we want to be integrated and go out and fix your integration sets to make sure that you can communicate with us this way." So to us, that didn't make any sense, right?

We understand that, you know, connecting to various SSPs, various different demand sources, they all have their own flavor, they all have their own integration process, they all have their own engineers who believe that they did it the best. So having one fixed way to communicate to a specific demand source just pretty much tells you that either you're not going to pass over all the information that they require, you're not gonna pass over things like metadata, things like parameters, just because, like I said, they're all unique in their own way.

So in Nimbus, what we decided to do a little bit more malleable. We decided to make Nimbus communicate the way each demand source was designed and built to. That way, we have now 30 to 40 different integration types but we understand and we know every single auction call is the most effective and efficient way you can do so.

Secondly is speed. As you guys know, time is money, right? Any impression that's not rendered is impression that's never monetized. So there is no point of having this auction, having this wonderful ad server, but by the time it gets to your user base, you know, that user has swiped past that creative and it never becomes a dollar. So what we decided to do is make our overall auction 500 milliseconds or less. We made sure of this by making, one, Nimbus more efficient, making it fast and making it effective, and also making sure our demand partners have their timeout set out at that specific rate as well.

At first, we got some feedback. They were saying that their demand needs more time, yada yada, not the case. We noticed that immediately, our overall performance has just skyrocketed because all the opportunity that was initially missed was now being rendered almost 100% of the time.

Optimizations. So this is probably one of my most favorite things when having your own ad server. So with optimizations, before, running on ad servers, running on mediation partners, you get data such as, okay, you have your fill rate, you have your CPMs, you have your win rate, and pretty much, maybe click rate and all that. But having your own ad server starts giving you some information you didn't even think of as publisher. All of a sudden, you get data. Data's the most valuable thing you can have, right?

Now you get trends, you have your users coming in to your app daily. You understand now that your users are…bid differently depending on the demand source. So now you have…if you're integrated within, let's say, Rubicon, you're integrated within OpenX or whomever, now you know John, from Wisconsin, he gets a overall eCPM of $13. So instead of, you know, having a generic floor that goes just across all of your inventory source, now you can start sending dynamic floors based on your users, based on the trends that your demand sources respond to you.

So if Rubicon's bidding particularly on this geo, you just make sure that you only pass geos to Rubicon for that user case. And f you're seeing John is only being…or is always being utilized at a $13 CPM, you never make that floor less than $13 for John. So you have all these opportunities now to start optimizing and making your ad calls that much more effective.

So realtime reporting. So this is a tool I think that every publisher should have, whether or not they have their own ad server or not. As you guys know, integrating new demand, integrating new tech, integrating anything will potentially disrupt your whole ecosystem. So you don't want to have to wait 24 hours to know whether or not your new demand source spiked your error rates, if your new demand source broke your entire ad stack, if your new demand source did exceptionally well and you can call them up and tell them, you know, "Thank you for all that revenue." But having realtime reporting is key. You don't want to have to wait for anything. You can integrate someone, and, in real time, know what they've been doing and if they belong in your ad server or not.

So SSP and DSP integrations. So these are some things that I noticed as a publisher when, like I said earlier, when you integrate through a mediation partner, all of a sudden, let's say, I want to partner up with A9. A9 does so well for me. I want them to integrate with someone like my mediation partner and then they return to me and they say, "Oh, sorry, we don't have a partnership with them, but here's some other SSPs that you're capable of integrating." So now, all of a sudden, you take the power away from me. No, I don't want to work with those SSPs, I know who works best in my ad stack and I want to have that autonomy to say who performs. So now, having your own ad server, you're the one, as a publisher, who says where your traffic gets sent to.

And rendering. So rendering is like this whole thing, that us, as a publisher, had no idea that this was a thing that needed to be done. We thought building our own ad server was really where it was at, and now, it's kind of like, do we have to outsource the video player, do we have to do it in-house? We chose the latter. So we decided to build our own video player and also render it in general, for static and [inaudible 00:07:44] just because we didn't want to have this full on auction handled in-house, but then outsource to the rest of it to someone else.

So having full control, full circle, of your overall performance is really key. Because now, we're also noticing that previous SDKs or previous demand sources, their render rates aren't even coming close to what we're seeing on out own video player or our own renderer. And that's also because we're close to our inventory, we're close to our app. We know how our app works. So we know how our users swipe. So we wanna make sure that rendering happens as quickly and often as possible.

Okay, great. So we built our own ad server, we made it effective, right, we did it in a specific way. What's new? What's next? So in 2019, we decided we gotta be better, we gotta be smarter. One way we did that is we switched over our entire kind of connection calls over to RTB. We're running on 2.5, which is the latest spec. I know 3.0 is out, but I believe it's in beta. I believe we'll be on it as soon as everyone is on it as well. What does that mean, right? What does that mean to be on Open RTB?

It means you get standardized calls. It means, like I said in the previous slide, when you have all these unique ways to communicate to different ad networks and demand sources, things get wonky. Things get lost. Some demand sources aren't getting all the data that they require and they can't really achieve that. So having Open RTB calls standardizes that, you can pass over any and all information in one call. And in addition, it's just so much more effective. Things like passing calls through JSON objects, having those JSON objects gzipped, meaning that they're compressed. If they're compressed, that means that they can be passed so much quicker down the pipes. And now if they're passed that much quicker, your latency decreases and your revenue increases.

In addition, you get requests with bid values. Like I said in our previous slide, having a true header bidder means that you can get all your bid requests in each auction in real time in microcents. So now, every single auction is placed on a header bidder and the highest specific…the highest CPM, the most value creative takes that auction every time. And no latency. This should actually be updated to limited latency.

If there are any operations people in the room, you guys know, if you're trying to optimize the most you can in a specific mediation platform, or a specific demand source, you have to make so many different line items calling different creative types, different CPMs, different geos. So with Open RGB, you have one call, just one call sends all the necessary data down the line. You don't have to have these convoluted ad calls and decrease in latency altogether.

So how it all works. I know you guys already know. [inaudible 00:10:31] header bidding is nothing new, but I did wanna show you a visualization on what it means. So first, you have a client-to-server, server-to-server call to your ad server. And in this case, it's Nimbus. So Nimbus then makes a request to each demand partner in Open RTB auction, meaning that you make one impression, it hits every single one of your demand partners simultaneously, and in 500 milliseconds or less, they return with the creative and the CPM. And then Nimbus, or your ad server, chooses that highest CPM, passes it to the client, and the client renders. Easy, right?

So what does mean? It means more competition. It means equal opportunity auctions. No one is prioritized. Everyone has the ability to bid and everybody has the chance to voice their CPMs or in their creatives. It means that you get static calls, video calls, native calls all at once. What does that mean? It means you get unified auctions. So if you have a specific ad unit that can handle video, static, native, banners, why send out different line items or why limit yourself to one creative type per impression when you can send all of them at once? Now, all of a sudden, your fill opportunity increases drastically. In addition, you can call any size your ad unit accepts. And like I said, instead of having a line item, or a 300 by 250, a 300 by 600, a 720 by 90, you can now have one call that calls any and every ad size that you as a publisher choose in the specific auction, and now your demand partner, that can't fill 300 by 250, great, they have a 300 by 600 that they can pass down the stream. The 300 by 600 gets into the auction, CPM gets compared to everyone else's, highest creative wins.

So controlling latency, and this is like the biggest thing, right? A lot of publishers are always asking me, and this is like, what, I know, makes them most excited, the fact that we decided to connect every single demand source via server-to-server connection. That means that no need for SDKs. And that means that you don't have to worry that your users are gonna require Wi-Fi connection in order to download your app. So it drastically decrease the size of your app and it really makes the connections be that more fluid.

Fixing creatives, this is also what kinda what separates the ad server now from being just like something that I just…is pretty dumb, just call something, render something, right? Now, we added a video validator. And what that means is that we allowed the client to do less things. So now, the client or your device, it only has one job, just to play. So what that means is that when you have an auction, and this is what I meant by the highest CPM with the most valid creative. So what we noticed is that yeah, you do everything right, your auction is handled. Your CPM is the highest, now you pass it down to the video player, and the video player, unfortunately can't support that MIME type, that's a lost impression.

So what the ad server does is it validates to make sure that every MIME type is applicable to that device. So if you guys know the default iOS video player is mp4. But so many times, we get creative types coming in that are 3gpp and great, that was a $30 CPM and they won the auction, but the device will never be able to play it. So it doesn't matter. What we do is we validate that MIME type. If it's an invalid MIME, we disregard that auction and we go to the next highest CPM.

Secondly is we do text-slow CDN connections. A lot of demand partners, they don't work as well as the other. So Nimbus then, they make sure that the CDN connection is fast enough that when we do pass the creative downstream, that it actually is quick enough to render on the user screen.

And also, human error. I had no idea this was a thing. But apparently, you know, you have an mp4, the creative MIME type, comes in and wins the auction, but the creative type, because someone in the back decided to write m4p instead of mp4, even though it is an mp4 creative, it's not gonna render, and it's gonna break. So we validate for human error, things like misspellings and all that. So I think that's pretty cool. I think that's something that takes a lot of load away from the client.

And security. And you guys know, security is a hot topic, and rightly so. You have so much user data coming in and you have the responsibility of a publisher to protect that data. So we do a few things now as well. We make sure that we protect both the traffic that goes out to our buyers, so our buyers are now more confident with the inventory that they're receiving. So we do things like enforcing user agent validation. What that means is we have user agent parser, which kinda detects bot traffic, SEO tools, feed readers, to make sure that everything is clean down the pipelines. And if something's off, we're warned about it immediately.

And the next two, we're isolating environment by rotating keys, two-factor authentication, and this is really for your user base. You want to make sure that no hackers can come in. You want to make sure that no one who's not supposed to see that data gets their hands on it.

So to recap, right, what does it mean? What does it mean to have your own ad server? It means that you add competition in your auction without causing bloated latency. You have a server-to-server header bidding solution, with no SDKs competing for the same inventory, and now you don't have any prioritization, you don't have anyone, you know, having a first or second look. Everyone has an equal opportunity to win. Lastly, you get insight on your own inventory. You start getting data, you start seeing trends, you start seeing how you can monetize…you can start being smart about passing that information, passing your impressions to the right demand source, instead of just blindly to all the demand sources.

Thanks. If you guys have any questions, I think we have a few minutes for Q&A.

Moderator: Excellent. Any questions? I don't have anything on Slido at the moment, but any questions about app monetization and the way that you've been doing it? We have one in back, excellent. I will pop over there, one moment.

Here we go.

Woman: Hi. I know we all love a homegrown solution. I'd be curious to know what you think the biggest benefit is from building your own ad server versus using something like DFP or [crosstalk 00:16:52].

David: Right. I hear this question all the time. So the thing is, like I said, I tried everything. I didn't want to build this. I wanted to have the easy route where I just plug it in and it works. Going this route is difficult. It took us a lot of time, a lot of errors to kind of face before we got it right. And I would say the difference there is the array of kind of things that I mentioned. So you have no SDKs, right? And you don't have DFP prioritizing themselves over any other demand. You don't have any ad server…you have your ad server that's actually running header bidding versus running a waterfall solution on the backend and then passing the highest fixed CPM and then passing it to you. You have the true header bidding solution. And now on top of that, you have unified auctions, right. Now you have an opportunity to fill at any creative type, any size, and just…

Don't get me wrong. Open RTB 2.5 or Open RTB in general is nothing new. DSPs and SSPs communicate this way all the time. They've been doing it for a while. The only difference is that SSPs don't give you, the publisher, the ability to communicate with them. They require that their SDK lives in your app, and there's like a bunch of reasons for them to do that. And for me, I would rather have control and I would rather have that communication direct to DSPs. So something that I didn't mention to you guys is because we're Open RTB and we're a publisher, it's unheard of, but we have relationships with DSPs direct. So there is no SSP supply-side fee. There is no SSP demand-side fee. Now you get a true CPM every single time direct from the DSP. And I don't think that you can get that from any other kind of solution.

Moderator: We have another question up front here. I'm just gonna run the mic over, one moment. And again, if you don't want to speak publicly, that is totally fine. You can use Slido and send us the question anonymously.

Man: Two quick questions. One, I think on the slide or an article I read said that you guys improved your eCPM from $2 to $28.

David: Yup. That's higher now. That was last year. That was my last presentation.

Man: Is that across…what ad user types was that?

David: Again, that's the beauty, right? It's a unified auction. So we have an ad unit that's flexible. So what we do is we have a 300 by 250 static competing against a 300 by 250 video placement competing versus a full screen [inaudible 00:19:30] show competing versus a standard native. And the beauty about it is because you're constantly getting bid backs, and I was having a conversation with some of my colleagues before, a lot of times, through Open RTB, they require a loss URL. What that means is that if they ever bid, let's say at three bucks, right, and then you never render it, that means that they bid and they lost the competition. So what they're asking for now is to provide that CPM that they lost to, so you provide back a CPM of like eight bucks, and they're like, "Holy crap, we should probably up our game." Their machine learning starts adapting these loss URLs, they're seeing that they're losing at specific CPMs, they have to increase their overall bid. So this competition now increases your overall performance in-house.

That's why header bidding is just so much more effective, and it's really the route to go, and anyone who's running on line items or anyone who's not running on header bidding, I mean, Open RGP's the next step. You should specifically be running on header bidding, just so you could pass over bid values in real time.

But yes, to answer your question, it's all of them, all of those creative types compete against themselves to have the final highest CPM no matter the creative, no matter the size, take that impression.

Man: Okay, second question is have you guys thought about productizing or open sourcing Nimbus?

David: Yeah. So the biggest thing was initially we had…we just did it for us. It was a Timehop only thing. To be completely transparent, if we didn't do it, we wouldn't exist. We were not making enough money to survive, we had to think creatively. So we built this ad server, and then we had this organic kind of conversation, these articles coming out saying, you know, "Timehop did this." So we had a slew of publishers jumping on board and they're like, "We want it."

So initially, we were saying, you know, "This is a Timehop only thing." Now, we've kind of opened up, having conversations and talks. So we have a few publishers now that are testing out Nimbus and it's working for them, to similar results as it's working for us. And the beauty about that as well is the more publisher impressions that go through Nimbus, the more opportunity for DSPs to start buying direct.

So another thing is that we would also give you a relationship with, you know, that specific DSP.

Moderator: Excellent. Any more questions? Very good. Thank you so much, David.

David: Cool, guys.

Moderator: You do win the award for best shoes of the show.

David: Oh, thank you.

Moderator: Excellent.