Deprecated: Hook custom_css_loaded is deprecated since version jetpack-13.5! Use WordPress Custom CSS instead. Jetpack no longer supports Custom CSS. Read the documentation to learn how to apply custom styles to your site: in /var/www/html/wp-includes/functions.php on line 6078
Technology –
Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

The SpaceX (and Tesla) masterplan (Part 1)

(This is a followup post to (now) four earlier posts on forecasting.

  • The first in May 2015 forecast both blimp-based and dedicated building-based drone deployments (later patented by Amazon);
  • The second in October 2015 largely predicted Elon Musk’s Tesla Masterplan Part Deux by 9 months
  • The third in July 2016 among other things correctly hypothesised the use of Model X falcon wings for future possible Tesla bus designs.
  • The fourth was yesterday on SpaceX telecoms plans. This is the expanded version.
  • I try to get it mostly right but I mainly love the idle speculation).

In the mode of Elon, I propose to outline what I think SpaceX will do over the next decade, how that intersects with Tesla’s strategy of building a sustainable energy future (combined with sustainable transport) and ultimately how that fits in with making humanity a multi-planetary species.

There are a number of moving pieces to this strategy and I can’t cover them all in one blog post, but I will try to distill it down to three distinct parts.

  1. Create a space company to reduce the cost of orbital delivery
  2. Deploy an orbital ISP and get billions of customers (The X in SpaceX?)
  3. Use meta-material antennas everywhere (including Mars), creating synergies with autonomous transport systems/sustainable transport

There is far more to this, and as ever with making forecasts like this, it carries lots of provisos.

Part one

This part of the plan is already well underway.

SpaceX continues to successfully recover its first stage when the flight configuration allows it. It is increasing its stock of used-at-least-once first stages. These first stages will also double as boosters for its heavy lift rocket the Falcon Heavy when it’s due to launch later this year.

Landed first stage Falcon 9.

The cadence for launches is already impressive this year, after a couple of launch failures. All of this contributes to reducing the overall cost of delivering payloads to orbit.

Over the coming decade you can expect that the cost of delivering anything to orbit is going to fall significantly. It will become more like catching a flight — and this will create an awful lot more capacity to deploy lots more vehicles. But what demand will meet the abundant new capacity?

Part two

Here is where things get interesting over the next 10 years. Up to the end of 2016 there were 1,459 satellites in orbit around the Earth. SpaceX have a backlog of launches in the dozens.

But you have to wonder: even if SpaceX captured 100% of the entire global launch market, would the margin — even with re-use — be enough to make enough cash to continue building expensive vehicles like the Mars Transporter? I would guess no — there’s less than 100 launches per year globally.

Even if SpaceX had 100% of the launch market, and the market was growing at say 20% per year, and SpaceX was making say, $30m profit per launch (a decent margin you’d guess), then that will still only be $3bn profit — hardly enough to build a fleet for transporting 1 million people to Mars and building a colony.

And if all this launch capacity and reduction in cost was only for Mars purposes, how does SpaceX propose to pay for the Mars Transporter(s)?

So what’s the alternative?

Build a global Internet Service Provider (ISP) that requires nearly 12,000 satellites to be put into orbit. Let’s call this new ISP ‘X’ for the purposes of this piece. (it may be a spin-off company from the launch part— SpaceX).

But how do you build a global ISP? First you reduce to cost of delivering satellites to orbit. Then you:

  1. Establish an operation in Washington State (a centre for space talent) with the purpose of mass-producing satellites as efficiently as possible using techniques like 3D printing and advanced manufacturing — similar to systems developed in rocket manufacture / battery manufacture / car manufacture. Satellites essentially become standardised rather than bespoke — reducing the cost of manufacture through economies of scale.
  2. Hire engineers specialising in building beam forming / phased array technology. Seems like Field Programmable Gate Arrays (FPGA) for in-orbit configuration on the fly and custom Application Specific Integrated Circuits (ASIC) would be needed (as they are with most satellite firms I guess).
  3. Integrate new beam forming and phased array technologies into the orbital vehicles. Propose a dual system of Low Earth Orbit (LEO) and Very Low Earth Orbit (VLEO) vehicles utilising Ka-band spectrum for communications between satellites and ground gateways, and Ku-band spectrum for communications between satellites and user terminals (and later possibly Ka to terminals too).
  4. Using the enormous, growing (and somewhat redundant) Falcon 9 launch capacity begin launching the mass-produced satellites into orbit. Aim to make the satellites as small as physically possible so that as many vehicles can be delivered per launch as possible.
  5. Scale it to 12,000 satellites in VLEO and LEO. Sell it to every human and business on the Earth (but do so gradually as you build your capacity). Charge high prices to start with and then lower prices as you scale. Also sell backhaul capacity via the LEO network — by using lasers in a vacuum to communicate between satellites in orbit you can radically reduce latency — arguably even in comparison to current land-based fibre optic cables.
  6. Build gateways (ie ways to direct internet from a ground station to the satellite network and vice versa) on multiple continents. Build the gateways close to internet backbones for maximum reduction in latency. You might only need a few gateways per continent.

Part three

But what does the product look like and how do you go market?

In the style of how Tesla launched the Model S, Model X, Model 3 and the Tesla Powerwall / Tesla Solar Roof, X will launch a product to sell access to the X network. It will be presented by Musk on stage as a huge leap forward in broadband communications. It will also be presented as simple, well designed and integrated.

What form will this come in?

  1. Within 5 years all Tesla cars will come with an optional extra. An integrated connection to the X satellite broadband network. These will be integrated into the roof (or trunk/bonnet — since electric cars have more available surface area than fossil cars) and come in the form of a non-mechanical meta-material antenna. Cars will be able to be in constant communication with the network, delivering fast broadband to every car in the fleet (the antenna can keep in constant communication with the network even when moving at speed). And here’s an important point: if Tesla want to have millions of autonomous vehicles on the road in the future they will need fast data solutions for their vehicles. It seems debatable whether the existing cellular infrastructure could handle the load.
  2. Within 5 years, Tesla will launch optional extras for its solar home product. It will be an integrated X “tile” within a solar roof. The tile will have a permanent connection to the X network. Essentially every solar roof could both be collecting solar energy while also acting as your internet access. Of course it will be sold stand-alone too.
  3. X will also sell the meta material panels across industry once the orbital network has expanded. This will include aircraft (no moving parts and flush with the fuselage, saving fuel), ships, government, military etc. This will also include backhaul selling to other networks.

If you think I’m crazy, take five minutes and take a look at the video below from January 2016.

Last year Toyota invested in the Redmond Washington-based (yes!) and five year old Kymeta (who include Bill Gates as one of their investors too).

The X network will look broadly similar, but unlike Kymeta, X will not only sell terminals — they will be building the entire ISP solution — end-to-end. And of course X will not just be limited to cars — it will apply to any location, moving or static.

Here’s some key slides from Kymeta’s presentation:

Indeed you could argue that X should buy Kymeta. Kymeta boast that their system could handle 1Tb of data per month per vehicle (seemingly using the Intelsat network).

If you believe that Tesla/SpaceX are not going to pursue this type of technology then I would argue that this is underestimating Elon Musk.

Towards the end of the talk Kymeta founder Nathan Kundtz even appears to have a dig at a possible SpaceX network.

With 50 [Intelsat] satellites each the size of a bus, and new satellites going into space all the time, this is not some network that will be ready in 2025, this is the network of now.

And let’s look at who SpaceX is hiring in Redmond right now:

Once you’ve built this system, and hopefully overcome regulatory hurdles; got your first 100,000 customers; built the largest ISP in orbit over time; integrated all of your vehicles to the network — then you can use that technology and kn0w-how and scale to deploy the same system to Mars.

You would do this in advance of the colony being established. It makes more sense to pre-position a communications network at Mars than it does to build it in tandem or after astronauts have already landed there.

In summary:

  1. Sometime this year or early next year Musk will announce the plan for the “X” network. He will outline a broad schedule for launches and coverage areas and include high-level pricing.
  2. SpaceX will seek permission for various bands, and seek to overcome any regulatory hurdles. It will meet resistance from incumbents.
  3. Musk will later announce the launch of Tesla vehicles with integrated antennas to access the orbital network. Tesla will launch home products for broadband and sell X via its Tesla showroom network globally. These products will integrate will with solar roof/solar storage products.

I’m happy to bet a whole €100 that this will, in large part, be attempted. Whether or not it’s successful is another thing.

And if I am right, Elon Musk owes me a few pints of Guinness 😂.

(Disclosure: I’m a *very* small shareholder in Tesla. I’m the founder over at Vizlegal (in Ireland!) where we’re building a global API for law — a sorely needed thing if you want autonomous machines (and even a Musk Mars colony needs laws too!). I’m on Twitter if you have any questions)

Will SpaceX become the world’s biggest telecoms provider? Probably.

By launching 11,943 satellites SpaceX will do to telecoms what WhatsApp/Facebook Messenger did to SMS and in doing so capture a $1tn+ business — and there’s fringe benefits for Tesla.

(This is a followup post to three earlier posts on forecasting. The first in May 2015 forecast both blimp-based and dedicated building-based drone deployments (later patented by Amazon); The second in October 2015 largely predicted Elon Musk’s Tesla Masterplan Part Deux by 9 months, the third in July 2016 among other things correctly hypothesised the use of Model X falcon wings for future possible Tesla bus designs. I try to get it right but I mainly enjoy the idle speculation).

NOTE: I have followed up this blog post with a yet more detailed product analysis here.

SpaceX launching Intelsat 35a


I was recently in San Francisco and had a very random number of drinks with two very friendly employees of US telco AT&T. As is often the case I turned the conversation towards autonomous vehicles, and more specifically two of Elon Musk’s companies, Tesla and SpaceX.

I was curious about how cars, such as a Model S, have much greater data connectivity needs than ever before. Right now, Teslas connect to AT&T’s network and it seems clear that data needs will only increase for data hungry vehicles that drive themselves. Already Tesla cars consume quite a few gigabytes of data per month.

Not only do vehicles like Teslas need high-end GPUs for some of their basic self-driving features, but they need good data connections for sharing data with Tesla, and for things like in-car infotainment systems.

As we move towards autonomy (specifically electric vehicle autonomy), cars will likely have even more data needs. Will passengers in autonomous Tesla Mobility vehicles expect Wi-Fi with fast broadband speeds? Will there be two, three or five people per vehicle? Remember all that time you had at least one human concentrating on driving? Well that person will instead be browsing the web while listening to a streaming music service.

This made me wonder: Why would Tesla would continue to bother with a middle-man like AT&T in the future?

Musk is a big fan of full-stack approaches to enterprise — so it seems logical to speculate that the growing data needs of autonomous Teslas will become a core issue Tesla will want to own or control (and perhaps charge for).

This brings me to SpaceX.

Musk often says he has a first principles approach to thinking. So let’s do it.

If you wanted to connect everyone on a planet to high-speed broadband, given technology available today (and not technology based on copper-based infrastructure that has been with us for decades), would you bother digging holes, laying copper, fibre, backhaul, exchange buildings, lay cable that crosses oceans, then on top of that build millions of cell towers with relatively limited range that span continents in order to connect mobile devices?

Probably not — particularly if there was an alternative available where you could grab internet from the sky.

Or as Patricia Cooper from SpaceX said:

“…the common challenges associated with siting, digging trenches, laying fiber, and dealing with property rights are materially alleviated through a space-based broadband network”

(And if you were the sort of person who wanted to build high-speed connectivity on a brand new planet — such as Mars — because the 1 million person colony there will probably want to have iPhones, you would certainly not have the approach of building a land-based communications network. You would do it in orbit. It’s much cheaper.)

As of the end of 2016 there were about 1,459 satellites in orbit around the Earth. SpaceX last month announced a plan to launch some 11,943 satellites — multiplying orbital vehicles by an order of magnitude.

Why so many satellites? Global broadband. Why is this interesting? Because it is completely independent of the switch-based POTS legacy we still largely work with (albeit we have been transforming it from a switch system to an IP-based network for decades now).

What will this new SpaceX satellite system look like? First is the LEO or Low Earth Orbit constellation of 4,425 satellites. Here’s the breakdown:

The second part of the SpaceX constellation will be the VLEO, or Very Low Earth Orbit network, comprising 7,518 satellites. By operating closer to the ground, SpaceX say this will both boost capacity and reduce latency in heavily populated areas:

SpaceX say this should give 1Gb broadband with relatively low latency (25–35ms because of the lower orbit than other space-based systems) to millions (or billions) of subscribers using relatively small user terminals with coverage across continents. They illustrate the relative coverage of the LEO and VLEO systems here:

Both these networks would be meshed to co-ordinate and provide redundancy and capacity, for constant near-global coverage. As you can see here, the higher dots are LEO and the lower ones are VLEO.

But I love the phrase SpaceX use in the document, because it’s part of the narrative companies like them and OneWeb are using about providing broadband to poorer countries or under-served rural locations:

SpaceX has designed its V-band system to meet the dual requirements of the world’s broadband demand — namely, connectivity for rural, remote and hard-to-reach end-users, as well as efficient, high-capacity connectivity at all locations.

In journalism this is called burying the lead.

What SpaceX are actually seeking is to replace every broadband and communications provider on the planet, by cutting out the middle man of land-based networks that stand between you and the internet. In doing so they will be essentially competing with every communications provider in the world — a business valued at over a trillion dollars. Forget about poor communities in Africa for a second: this is a pitch to replace physical fibre/cable connections in modern industrialised economies.

But a few questions arise from this including the big one: mobile phones. Will the plan be to have mobile phones work directly with satellites overhead? Is that even possible? Or will there be a hybrid approach — provide broadband to physically static locations and work from there?

Which brings me back to Tesla.

Musk was keen to announce recently the launch of the solar roof product — solar cells embedded in most of the tiles on your roof. This roof would in turn be connected to a Tesla battery, storing energy during the day for use at night (and perhaps for your electric car too) — and leading to relative energy independence for many users — and a distributed grid.

But this raises another interesting hypothesis in my mind. OneWeb, a rival to SpaceX for this race for global internet, have shown their prototype device conceivably connecting schools and medical centres in Africa:

What’s interesting about this? The terminal on the roof is:

  1. Solar powered — so independent of the energy grid
  2. Doesn’t just provide Wi-Fi — it has 3G and LTE built in too.

So what would I do if I were in Elon Musk?

  1. I’d sell SpaceX user terminals to Tesla Energy customers. Each solar roof (or roof with Tesla solar panels) would come with an optional extra — solar powered high-speed broadband (in a form perhaps similar to OneWeb, or even under your roof). Your internet would become independent of the energy/telecoms grid, inexpensive (SpaceX reckon $200) — and de-centralised.
  2. I’d make each device at a user’s home a base station for providing Wi-Fi, 3G and 4G coverage. I’d make the offering so cheap that demand would be enormous. (Each base station may not have a huge range given its size and power consumption but one could imagine a mesh system working in a co-ordinated fashion). If you can get say, 50m homes between the US and Europe and charge $50 a month, that’s a clear $2.5bn in revenue a month — while utilising a relatively low maintenance satellite network.
  3. With enough customers in high density population centres my own customers would be paying for the roll out of SpaceX global mobile broadband infrastructure — I’d cut out the middle man of current telecoms incumbents that rely on land-based cables and towers. (I might also build some towers too, but this would be much cheaper because they would be also solar powered and would not need a physical connection). (note: another fringe benefit of the Boring Company might also be laying fibre). In addition I’d be selling backhaul connectivity since lasers in space can send data faster than through cables.
  4. I’d then also have a global infrastructure for Tesla vehicles to be permanently connected to space-based broadband (depending on how phased arrays evolve this could be that each vehicle is connected constantly to a satellite, or it uses a mixture of that plus terrestrial devices mentioned in 2).
  5. Use the enormous revenue generated from displacing incumbent telecoms players (A back of the envelope calculation says the major telcos combined value is over $1 trillion+) and take what I’ve learned and apply the same technology to investing in, and rolling out, Mars-based orbital telecoms infrastructure.

SpaceX plan to launch their first prototype later this year and the first satellites for the constellation in 2019 with a target to complete the network by as early as the mid 2020s (but likely later). And with their success in re-usable first-stage rockets, they have the means and will to meet an ambitious satellite launch schedule, and at a cost that is likely to continue to fall.

It also might bring about the death of land-based networks such as AT&T (current market cap $223bn).

And perhaps even more ironic — SpaceX is charging existing telecoms and TV providers such as Iridium lots of money to launch satellites. But SpaceX’s longterm strategy could be to use the profits from these launches to replace their own customers with a more compelling, fast and global space-based mesh network and offer connectivity directly to every citizen on the planet. So like solar roof customers paying for SpaceX infrastructure, so too would existing communication companies pay for their own possible demise.

So in summary—

  1. Start with first principles
  2. Go fully vertical
  3. Choose distributed over centralised
  4. Let your customers pay for your infrastructure roll-out

(Disclosure: I’m a *very* small shareholder in Tesla. I’m the founder over at Vizlegal (in Ireland!) where we’re building a global API for law — a sorely needed thing if you want autonomous machines (and even a Musk Mars colony needs laws too!). I’m on Twitter if you have any questions)

Working to fix the YouTube hate speech problem

Last week The Times in London published a story concerning hate speech videos and the advertising surrounding them. The story by investigations editor Alexi Mostrous began:

Google is to be summoned before the government to explain why taxpayers are unwittingly funding extremists through advertising, The Times can reveal.

The Cabinet Office joined some of the world’s largest brands last night in pulling millions of pounds in marketing from YouTube after an investigation showed that rape apologists, anti-Semites and banned hate preachers were receiving payouts from publicly subsidised adverts on the internet company’s video platform.

David Duke, the American white nationalist, Michael Savage, a homophobic “shock-jock”, and Steven Anderson, a pastor who praised the killing of 49 people in a gay nightclub, all have videos variously carrying advertising from the Home Office, the Royal Navy, the Royal Air Force, Transport For London and the BBC.

Mr Anderson, who was banned from entering Britain last year after repeatedly calling homosexuals “sodomites, queers and faggots”, has YouTube videos with adverts for Channel 4, Visit Scotland, the Financial Conduct Authority (FCA), Argos, Honda, Sandals, The Guardian and Sainsbury’s.

At the end of the piece was Google’s response:

A Google spokeswoman said that the company had “strict guidelines” relating to advert placement and that in the vast majority of cases its policies “work as intended”. The company “doesn’t always get it right and sometimes ads appear where they should not,” she said, adding that it would make changes to policies and brand controls.

Since the publication of the story many brands and advertisers have pulled their ad campaigns pending clarification from Google.

Mostrous tweeted an image of The Times editorial that went with the story

This tweet led to an interesting conversation between Alexi, Benedict Evans from Andreessen Horowitz and later Rob Kniaz of Hoxton Ventures. The TL;DR of this discussion is as follows:

  1. Alexi: Google must remove hate speech from YouTube and [to quote from the editorial] there are no technical barriers to doing so.
  2. Benedict: ‘There are no technical barriers’ is gibberish & manually verifying billions of hours of content per day is impossible.
  3. Alexi: Google should be more pro-active and less reactionary. It tends to react to flagged content rather than rooting out extreme content itself.
  4. Benedict: [it] would need speech recognition on every video. And scanning all videos for text. That’s not easy at all.
  5. Alexi: Why not start with 200 people and pro-actively examine content?
  6. Benedict: Your basis for claiming a technical solution being easy is untrue.
  7. Rob: As an ex-Googler I can confirm it’s not easy.
  8. Benedict: There is a problem, Google should do more, but claiming it’s easy is wrong. You can’t use people to ‘edit billions of hours of video’ either.

There are two issues at play from the original story; one is that such extreme videos are on YouTube at all; and the second that advertisements from premium brands are appearing adjacent to this type of content — allowing publishers of such content to make money from public and private sources of ads — often without the knowledge of the brands themselves.

I’m inclined to say that both Alexi and Benedict are correct and wrong, but for different reasons.

I spent many years at Storyful (which was acquired by The Times’ owner News Corp) with breaking news content on YouTube, and working with my colleagues to find original — often graphic — content, working on YouTube for the web first, and then via the API to find content that is real/original, re-uploaded/copies, copyrighted content, or what is often referred to now as ‘fake’ content. This would often involve millions of API calls to find and verify the content we needed.

Of course it is the case already that YouTube does employ/contract people to deal with content — YouTube Policy. A quick look at LinkedIn suggests approximately 400 people working on this problem at YouTube — though Google generally does not share the actual number of staff working on this team.

I’m going to start at this problem from how it was articulated by Benedict: that is technically extremely difficult or impossible to vet billions of hours of video per day.

This is undoubtedly true, but I think it’s also a bit of a straw man argument.

The first question is: do billions of hours of videos need to be vetted algorithmically or manually to help solve this problem? I’d say no.

YouTube is built on two things; content and the accounts that upload that content. If you want to build a system to vet hate speech, for example, you start with accounts that create the content, not the content itself. From an algorithmic standpoint this is the lower hanging fruit. And if you want to start with even lower hanging fruit, you start with the known creators of extremist content.

In order to create a YouTube account, you need to create a Google account. This usually involves giving a real name/username and a real phone number to confirm it (though this is not obligatory). This is the starting point and here are some questions to ask:

  1. What accounts are uploading content that is being repeatedly flagged as hateful or in breach of YouTube policy?
  2. Before even getting into the possible whack-a-mole problem of sock puppet accounts, who are the repeat offenders and what content are they uploading? Are there other websites/social links to those same users?
  3. What data does YouTube collect at the point of account creation? Is the barrier too high or too low for account creation?
  4. When IP addresses are collected during account creation processes what happens? Yes, some users will use VPNs etc, but there are several steps that go from user x creating an account, to uploading a video, to then that video being removed. One could imagine lots of stuff being done here.
  5. If a video is flagged and removed and then re-uploaded, is it caught automatically and flagged using the YouTube CMS? (YouTube’s backend systems already detect duplicate content using a combination of audio and video matching).
  6. What other data does Google have outside of YouTube? (given that they are separate commercial entities). If a hate-speech website that has already been flagged as associated with questionable videos is using Google Analytics for example, are those signals recognised? Is there a flag to say: if website x embeds any video it’s an automatic flag on YouTube’s CMS as likely to need further vetting?
  7. Has the team in YouTube Policy expanded in line (on any basis) with the explosion of uploaded content on a per/billion hour video per/day basis? I would guess not. If not, how can they be expected to perform the same function as say, five years ago?
  8. If technical solutions are being employed to support the policy team, as I expect they are, are they enough? Recent evidence suggests no.
  9. Clearly spam accounts are an enormous issue at YouTube, as they also create a server cost for a) hosting the videos and b) playing them. Understanding the difference between spam and non-spam accounts is enormously difficult. But that doesn’t mean that you can’t create simple filters to start you on the road to vetting what is likely to be extreme content.

Therefore: a possible checklist upon upload of a new piece of content by either a) a brand new account or b) an account with which there were issues before (in whatever order is the most logical)

  1. Is the account new from a brand new user who has never posted to YouTube before, or is someone creating a new account who already has other accounts or has been banned? (Plus some magic sauce about the browser/OS/IP address etc creating the account).
  2. Has the account been around for a while? Has the account uploaded flagged content before?
  3. If the account has been around for a while, have algorithms been used to mine that account for: a) all comments ever posted beneath every video ever posted containing a mixture of hate speech/keywords/keyphrases. b) Has NLP been run across all comments to gauge the video content? c) Have algorithms been employed to score accounts based on this easy-to-obtain text content? d) Has SNA been used to graph the relationships of the commenters that surround extremist content? Are they uploaders themselves?
  4. Does a new video posted by a freshly made account contain flagged words or phrases — not in the video itself — but in the video title, the video description or in the earliest comments associated with the video? Are there links from the account or the video to sites that are flagged? Was the HTML of the website in question mined for keywords too (using Google Spiders for example?).
  5. If the video contains a bullshit title, and a nonsense description, but the video itself is questionable, how do you detect it? Are comments on or off? Where was the video embedded, if anywhere? Is it a known website? If it wasn’t embedded what can be learned from the video via other signals (before getting into audio analysis).

I’m sure the smart people at YouTube have thought of all of these things, however one of the perennial issues that affects YouTube is its relationship with Google — ie they are not the same thing. So it can be hard to get both companies on the same page, despite being from the same company.

It is also clear that the problem is not necessarily that every video uploaded by every person has to be checked, as Benedict seems to argue. What can happen at a technical level is outlined above — and more.

At Storyful we had built enough intelligence on top of YouTube to know what known account was likely to upload content of a real world event before the account even did so. We’d also know whether that content was likely to be graphic in nature before watching it. And we’d also have some idea of the reliability of the account.

And it the account was new to us: we’d have a fair idea whether the account was a sockpuppet account, a legitimate account, or a re-uploader, using signals available through YouTube’s own API (e.g. account creation date, related accounts, number of videos already posted).

And that was five years ago.

Alexi and Benedict are both right that YouTube could be doing more. Alexi is right that they could be doing a helluva lot more. Benedict is right that it’s not technically easy to mine billions of hours of videos in realtime — but that’s not necessarily the problem either.

The problem is this: YouTube has a policy on what videos can and cannot go on its platform. It has likely erred on the side of letting more content through than it should. It should re-consider.

And as for the other problem of ads being displayed next to extremist content: brands want to be assured that their ads are not associated with hate speech — by working to solve the problem above, YouTube also benefits by being able to assure brands to a greater degree than before that their ads are not showing next to such content (a YouTube CMS equivalent for where ads show).

From pull to ‘stream’ economies

I was interested to read Ben Evan’s recent take on the “Facebook of eCommerce”. He concludes:

That kind of scalable automation, though, could also go in completely the opposite direction for some things – away from any kind of decision at all. You put an Amazon Dash on the machine, or perhaps it can measure what you’re used and re-order by itself, and so you in effect subscribe to the product, and once done you’ll probably never bother to change brand. Or, say to Siri or Alexa or Google Assistant ‘Hey, order some more soap powder’ and the same brand is added to your next delivery. (And in both cases your choice of channel is just as now locked in as your choice of soap powder, once you’ve set the default.) Either way, an impulse purchase in one of 2 or 3 retailers you might have stopped in at, based on real-estate portfolio on one hand and eye-level placement and brand equity on the other, shifts to auto-renewal or a natural language parser. Given that P&G and Unilever’s combined ad budget is larger than the global revenue of the recorded music industry, this means that subscription soap powder could be a much bigger deal than subscription music. What will you have to pay to be Google Assistant’s default choice of dishwasher tablets?

It’s a well made point. But I think it could be looked at from another angle.

One of the core philosophies we developed for building systems at Storyful was a switch away from search-based systems to stream-based systems. I always felt that one of Twitter’s core innovations was its Stream API. Unfortunately it remains one of the few publicly available stream APIs out there (and to get it at any scale you need GNIP too).

When you’re trying to detect signal in noise, streams of data that you can filter can work incredibly well. Too many APIs, like for example YouTube’s, were based on the idea of repeatedly polling it to ask the same or similar questions of their data. Asking “any new videos uploaded containing the word ‘x'”, millions of times a day is not very efficient. (There were some attempts to streamify YouTube’s data using PubSubHubbub in the V3 API but this isn’t quite the same)

Rather, just getting the raw ‘stream’ data to manage and act on ourselves was far better – hence we spent a good deal of time converting REST APIs into Stream ones for our own purposes (using lots of calls) – and then building secondary systems and algorithms on top of those streams to detect events, anomalies, patterns and so on – across multiple platforms.

The same could be said of what Ben hints at – a switch away from user intent, ie “search“, or “GET”, to deliver, stream, or ‘push’. Google and Amazon are search systems. A user has to go find stuff and order/click it, usually in discrete transactions. Based on your behaviour the system might suggest other products or results that might interest you. The infrastructure that Ben mentions is what I would describe as streaming products. I subscribe to a “stream” of washing-up powder and it just arrives when required (based on either censors or figuring out on average how frequently I use it up).

The obvious next step from these kind of rudimentary streaming products is smarter streaming products. That world is one where I divest most control over rudimentary purchases entirely to a digital assistant (and by mine, I mean one designed for me, by me, that’s independent of platform or service). One could imagine entire industries built on trying to convert me one from one product “stream” to another, and users arbitraging en masse to receive either greater discounts, or alter the behaviour of producers. I assume this is where things like Jet are going.

The system will figure out what I need, when I need it, and even what I don’t need, but probably want. Then it will stream it to me. And this goes for digital products as much as it goes for physical ones. (An odd logical extension of this will be machines ‘advertising’ and ‘negotiating’ with other machines to change streams on my behalf).

Push, not pull. Streams, not requests.

Elon Musk’s sleight of hand

[cross posted from Medium].

Like many people, I’m a fan of Elon Musk, the CEO of Tesla, SpaceX and Chairman of Solar City. So much so that I’m nerdy enough to listen to the quarterly conference calls of Tesla, and keep a close eye on the movements of each company.

Watching Tesla launches, like the recent Model X and Powerwall announcements, all remind me of watching Apple and Steve Jobs product launches back when it was still considered fanboy(ish), and not a pre-requisite for people working in tech or journalism (ie anytime pre iPhone in 2007).

Musk’s presentation style is not as polished as a Jobs show — but he manages to pull it off in a slightly awkward, if endearing, manner.

Indeed, like back then with Jobs, today many people have no idea who Musk is — he has yet to meet the Jobs levels of fame.

However, beneath some of the recent announcements are I believe some more fundamental things at work. Clearly everything I write is only as an interested observer, and is certainly not based on any fundamental research. I’m as in the dark as everyone else about Musk’s future intentions — but I do enjoy exercising my brain on what’s possible or probable.

Before we begin, keep in mind throughout Tesla’s stated goal: “To accelerate the world’s transition to sustainable transport.” It’s not to make the coolest looking electric cars.

This week Musk launched the long awaited and much delayed Model X — the SUV followup to the incredibly well reviewed Model S sedan. But during the show, Musk almost downplayed features of the Model X that, within the right circumstances, are in my view nothing short of revolutionary. Some features already exist in the Model S — but I believe this new combination is a step in a new direction.

Let’s start with the first example.

A dozen minutes into the launch of Model X, Musk says

“So let’s move on to the car itself. What’s cool and fun about the car? Doors & Windows. So. You’re obviously familiar with the Falcon Wing door. What we also have is an Auto Presenting front door. So what it will do, it will triangulate my position and detect that I am moving towards the front door. It will open the front door. Without me touching anything. I will sit down, and it will close the door. Like an invisible chauffeur. (He then laughs to himself in the car)

It’s a cool and fun feature. But was it a feature added to the car because it was cool and fun? It seems like quite a bit of effort just so a human doesn’t have to touch the handle of a door and close it after them. It’s like a first world problem of first world problems.

And it’s well beyond “fun” when you’re building any expensive complex device such as the X — which Musk has previously described as “the most difficult car in the world to build”.

But onto to the second example.

Later in the presentation Musk focuses audiences on how the Falcon Wing doors are a wonderful innovation and “look cool”. But the main crux of this innovation, Musk appears to argue, is the ability for parents to get full advantage of the second and third rows of the Model X — without the discomfort of “cantilevering” themselves and their kids seats as they would with normal SUVs.

Also during this demonstration (left), Musk “presses the button” (he actually says those words) so that the second row seats move themselves forward electronically. He then gets in the third row, to demonstrate the space and ease of ingress.

I’m now asking myself a number of questions during this demo — which only grow when Musk moves on to talk about the Falcon doors.

Which leads us to example number three.

In the next set piece, he shows how easy it is to get into the car via the Falcon Doors when two other vehicles are parked directly alongside the X. Ostensibly, the rationale for this demonstration was again the scenario of perhaps parents at a shopping mall, trying to manage their shopping and their kids — and some rude people parking beside you. The Falcon Wing doors sense the proximity of the nearby cars, and still open with ease, again allowing for ease of ingress for humans.

Which brings us to example number four:

Musk, almost in passing, mentions the extra room around the rear seating area. Here he outlines how wonderful this feature is:

“Probably the best-looking second seat — if that’s a superlative — ever. But it actually provides more functionality because you have a flat floor and you can stow something. So if you’ve got a backpack, or a laptop, or a handbag you can stow that under the seat, instead of having it at your feet. So it actually provides utility as well as aesthetics.”

Except that later in the presentation, Musk and his team demonstrate the enormous overall storage capacity of the X — so I’m left wondering why emphasise the extra stowing feature under the rear seats?

Lastly is a feature that wasn’t actually presented — but is a feature still under development — the “snake”. This was demoed plugging into a Model S earlier this year, but will clearly be compatible with the Model X too, whenever it becomes available. Essentially it is a charger for the car that recognises when a vehicle is present and plugs itself in, without the drivers having to get out and do it themselves.

But when I pull these five things together I don’t see features that are being built or added because they are “fun”, or because they are designed for frustrated parents in shopping malls with more luggage than any family in the history of the world. How much did each of these features cost in both time and money for Tesla? I wonder.

No. None of these features have anything to do with building conveniences for humans too lazy to open doors with their hands, or indeed for parents squeezing between cars.

They were built for something else — and this is Musk’s sleight of hand.

All of these feature were built for one reason — a self driving future combined with an entire self-driving mobility platform. The Model X was built to be either the ultimate self-driving taxi, or the ultimate human/self-driving rental car — or both. Or as Musk almost laughingly hinted during the presentation — an invisible chauffeur will be doing all the work.

1. A front door that opens when you approach it and closes itself when you get in — because it’s fun? No. A self-driving car that arrives to collect you and opens its doors when it detects your proximity based on your watch/mobile device nearby (plus the sensors).

2. Electronic seats that move forward to make the lives of parents easier at the touch of a button? No. A software update will allow the seats to configure themselves for passengers arriving to get into a car where the doors open themselves (Uber – but you tell it how many people and the car gets ready for the group).

3. Ease of ingress and egress for humans in the Model X because of Falcon Doors? No. The doors don’t exist for frustrated parents — they’re doors designed for a self-driving taxi/rental mobility platform.

4. More storage under the rear seats because you need more of it, and because you can (down to the space that electric cars give you)? Yes. But when Musk uses the word “stow” I think airline. And when I think airline I think passengers. And when I think Model X I think taxi — with lots of room for your bags — with no driver in the front seat.

5. A snake that extends to charge your car because it saves your lazy ass from having to get out and plug it in yourself? Yes, but if the car is driving itself it’s going to have to be able to reverse into a station and commence charging — without the presence of a human.

If I’m correct — and I think I am — the future for Model X owners won’t involve them being the only drivers of their own cars. It will involve them renting out their cars to everyone else for a price — with Tesla taking a cut — and the car driving itself. As Musk so often says, cars spend most of their productive lives sitting unused in people’s driveways. Which is crazy for such an expensive piece of hardware.

Model X will be a self-driving car with doors that open when you approach, seats that configure for the number of passengers who can then easily ingress and egress through Falcon doors, with lots of in-car stowage available, that runs on batteries in the floor charged by solar fuelled battery packs at supercharger stations (and elsewhere).

How will Uber, Hailo, Hertz, Avis, Enterprise, Budget et al compete with this? It’s not exactly clear to me. All of those firms rely on fossil-fuelled cars and humans to function. Both involve high costs (financially and environmentally).

Tesla vehicles run (or will ultimately run) on freely available solar energy — for no charge to its owners at supercharging stations.

And one has to imagine that the Model X has much if not all of the hardware necessary that — should a certain over-the-air update arrive at some point in the future — then the thing will just drive itself around.

I’m not the first to speculate on what might be called “Tesla Mobility”. Adam Jonas at Morgan Stanley recently asked Musk directly during a conference call exactly this type of question. Musk decided it was best not to comment. And this was before we saw the Model X launch.

And remember: “To accelerate the world’s transition to sustainable transport.” Yup, that’s what Tesla Mobility would be, if Tesla can pull it off.

At the very least, the next five years (not the next ten, this will happen faster than we think), will be very interesting.

(Disclosure: I’m a *very* small shareholder in Tesla and Solar City. I’m the founder over at Vizlegal (in Ireland!) where we’re building a global API for law — a sorely needed thing if you want autonomous machines to know what human laws to obey (and even a Musk Mars colony needs laws too). I’m on Twitter if you have any questions!)

Facebook moves the goalposts

For all the Facebook users out there:

You hereby grant Facebook an irrevocable, perpetual, non-exclusive, transferable, fully paid, worldwide license (with the right to sublicense) to (a) use, copy, publish, stream, store, retain, publicly perform or display, transmit, scan, reformat, modify, edit, frame, translate, excerpt, adapt, create derivative works and distribute (through multiple tiers), any User Content you (i) Post on or in connection with the Facebook Service or the promotion thereof subject only to your privacy settings or (ii) enable a user to Post, including by offering a Share Link on your website and (b) to use your name, likeness and image for any purpose, including commercial or advertising, each of (a) and (b) on or in connection with the Facebook Service or the promotion thereof.

In other words, even if you delete your account, all the content you uploaded remains the property of Facebook. Careful now.

Google Earth 5.0

Pretty awesome stuff. They’ve finally added ocean topography. Also the following rather cool features:

* Historical Imagery: Until today, Google Earth displayed only one image of a given place at a given time. With this new feature, you can now move back and forth in time to reveal imagery from years and even decades past, revealing changes over time. Try flying south of San Francisco in Google Earth and turning on the new time slider (click the “clock” icon in the toolbar) to witness the transformation of Silicon Valley from a farming community to the tech capital of the world over the past 50 years or so.

* Touring: One of the key challenges we have faced in developing Google Earth has been making it easier for people to tell stories. People have created wonderful layers to share with the world, but they have often asked for a way to guide others through them. The Touring feature makes it simple to create an easily sharable, narrated, fly-through tour just by clicking the record button and navigating through your tour destinations.

* 3D Mars: This is the latest stop in our virtual tour of the galaxies, made possible by a collaboration with NASA. By selecting “Mars” from the toolbar in Google Earth, you can access a 3D map of the Red Planet featuring the latest high-resolution imagery, 3D terrain, and annotations showing landing sites and lots of other interesting features.

If you haven’t already got it, you can download Google Earth 5.0 here, or indeed upgrade from 4.3.

How many people were at the inauguration?

I, like many others, was asking that question throughout the day. Being on the ground it felt like Croke Park times 20. Slate asks how best to reach a figure:

Thanks to advances in aerial digital photography and computer image-processing, it’s now possible to get a fairly exact head count—without a magnifying glass. As Farouk El-Baz of Boston University explained in a 2003 Wired article, the best way to obtain an accurate image is to fly over the assembly at peak time and take a digital photograph (resolution 1 foot per pixel) from 2,000 feet or less. Using satellite images, an Arizona State University professor calculated that about 800,000 people attended the inauguration Tuesday—considerably fewer than the AP estimate (based on photographs and comparison with past events) and less than half the Washington Post number (based primarily on security agencies on the ground).

I think Slate is looking at it from the wrong perspective. To me the core issue is mobile networks, not digital images. Why don’t the US cell network firms, which deployed extra cell towers all over the mall, just release the data on how many people had cell phones in the area and then use this information for some free publicity?

Perhaps certain people like myself might have more than one phone on them, but most won’t. I imagine it would be fairly easy to tell how many people were on the mall by counting the number of active cell phones in the area.


The new website went live today. I prefer it over the previous incarnation. I haven’t started uploading photos (not that there was much to upload, perhaps some Georgia/US election ones).

Not heard of it? It’s a British-based website for photographers to upload and sell their photos, sharing revenue with Demotix (who sell the photos for you). More here.

Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Undefined variable: todo_styles in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120

Notice: Trying to access array offset on value of type null in /var/www/html/wp-content/plugins/bwp-minify/includes/class-bwp-minify.php on line 3120