The perils of polling: C-11, YouTubers and Abacus Data

Conservative Senator Leo Housakos asked Abacus Data to disclose a YouTube sponsored public opinion poll

May 2, 2023

It can’t be easy being a public opinion pollster. That’s especially true when it comes to taking soundings on detailed government legislation.

Polls are good at capturing the public mood, our gut instincts if you will. In May 2022 Nanos released a poll showing broad support for the federal government’s intentions to pass three “Internet” bills: the Online Streaming Act C-11, the Online News Act C-18 and an as yet untabled Online Safety Act. But it would be foolish to mistake that broad support for specific endorsements of the Bills as drafted.

Then there is the fraught relationship between pollster and client. The political actor that commissions a poll hopes for a particular outcome while the pollster must maintain its professional standards without alienating its client.

That is why we, the consumers of these polls, need to scrutinize very closely what questions have been asked and contemplate the environment in which answers are given.

On October 14, 2022 while the Online News Act C-18 was before the House of Commons, Google published a poll conducted by Abacus Polling that appeared to replicate its criticisms of the Bill through polling results. In a blog post, MediaPolicy.ca suggested the questions were so leading and torqued that the answers were unreliable.

On October 25th, Abacus Polling CEO David Coletto appeared at the Senate hearings on Bill C-11, the Online Streaming Act.

Lead Conservative Party critic Leo Housakos asked Coletto if he had done any polling on the controversial elements of the Bill, in particular the CRTC’s powers to compel social media platforms to recommend more Canadian content. Coletto admitted there was a poll that he had conducted on behalf of Google-owned YouTube but he was not authorized to release it.

In fact, the poll was never publicly released but Housakos asked YouTube to submit a copy to the Senate, now duly posted on the Transportation and Communications Committee’s website.

Here is how Abacus described findings that bolstered YouTube’s case against C-11:

Our research found that video and music have become ubiquitous and essential in the lives of Canadians. Users are overwhelmingly satisfied with the services they use. This positive attitude is driven by several factors, including the content, choice, convenience, and value-for-money offered by today’s streaming services. Access to Canadian content was rated far less important a factor to the experience.

The survey included a section where respondents were asked to react to different scenarios. Each scenario presented a hypothetical change to today’s streaming experience (e.g. Canadian content prioritized in video queries or in content recommendations). Many respondents had negative reactions to the changes. In some cases, their reaction was so strong that users indicated they would cancel the service or go as far as using a VPN to get around the changes.

The study also found that most Canadians are concerned about Bill C-11, the Online Streaming Act. They think it should be up to users to determine what content they see. They feel the Bill’s interventions could negatively impact their access to music and video streaming services.

The poll found that 89% of Canadians stream music or video (not just on social media), compared to 66% consuming content on cable TV, 68% over YouTube itself, and 75% on streaming services other than YouTube. Not surprisingly, 81% of poll respondents described access to streaming as “essential or somewhat essential.”

So far, so expected.

Then the poll jumps straight in to some questionable questions.

• First: How willing would you be to provide information about your residency or nationality in order to upload content to platforms like TikTok or YouTube. [Result: 35% willing, 58% hesitant].

Here’s MediaPolicy.ca’s take on that. The question is aimed at measuring public opinion on the wisdom of requiring Canadian YouTuber creators to declare their Canadian nationality for the purpose of recommending Canadian videos or music. That’s not explained by the question. In addition, the question was asked of all respondents, not the one per cent of Canadians who are YouTube ‘creators.’

• Second: When it comes to which content appears in streaming services, playlists and recommendations, would you rather that content be determined by:

What the streaming service algorithm determines is most relevant and accurate;

or, Government legislation and regulations first, followed by what’s the most relevant content.

Framed this way, the invitation is to choose between YouTube’s unregulated curation versus unspecified government-sponsored content as the top recommendations —relevant or not— before accessing YouTube’s relevant search results. Why anyone, let alone 15% of respondents voted for that Orwellian nightmare is unclear, but to suggest that’s an outcome of Bill C-11 is highly argumentative.

***

Next up in the poll are hypothetical scenarios of how the social media audience might react to YouTube altering their search recommendations as a result of CRTC demands for discoverability of Canadian content.

The question is framed as how likely is it that the respondent would ‘cancel the social media service’ —how does that apply to free service like YouTube and TikTok?— or use a VPN to view unregulated platforms from an American IP address?

• Scenario #1: MUSIC SEARCH: Imagine you were searching for a specific artist or type of music on a streaming service. When the search was complete, 25% of the results appeared primarily because the streaming provider was required to show you other search results.

The MediaPolicy.ca comment: a properly framed question should have specified either “….show you other search results relevant to your inquiry from Canadian artists” or “…show you other search results regardless of relevance because they are Canadian?” Many poll respondents may have assumed the latter.

• Scenario #2: PLAYLIST RECOMMENDATIONS: Imagine you are listening to a playlist of songs or a channel of music dedicated to a particular genre. During your listening experience, 35% of the songs are played primarily because the streaming provider was required to play you certain content.

MediaPolicy.ca: Again, the question fails to distinguish between relevance or non relevance of search results.

• Scenario #3: USER-GENERATED CONTENT #1: Imagine you are on a site that shows user-generated content like YouTube, TikTok, or Instagram and you are searching for something related to an international event or cultural practice. When the search came back 25% of the results and many of the top results appeared primarily because they were Canadian content.

MediaPolicy.ca: Again, the question assumes YouTube would include government-mandated irrelevant recommendations to this search inquiry. The question assumes that YouTube, which need only demonstrate to the CRTC an overall compliance with discoverability, would choose to provide exactly the opposite of the search inquiry. 

• Scenario #4: USER-GENERATED CONTENT #2: Imagine you are on a site that shows user-generated content like YouTube, TikTok, or Instagram and you are searching for something related to a do-it-yourself project. When the search came back 25% of the results and many of the top results appeared primarily because they were Canadian content.

MediaPolicy.ca: Again, the question assumes YouTube will provide Canadian results that lack relevance to the search inquiry. The other issue to consider here is whether ‘DIY’ is a cultural area where CRTC would expect more effort at discoverability. 

***

At this point in the poll, the working assumption embedded in the questions has been over-regulation of content through more (how much more is undetermined) and irrelevant recommendations of Canadian videos.

So Abacus’ next question is very interesting: 

To what extent do you agree or disagree with the following statements?

• It should be up to users to determine what content they want to see (Results: 93% agree or somewhat agree)

• I do not think government should decide what content aligns with Canadian values (84% agree or somewhat agree)

• I am not confident that the government has the capacity to determine what content aligns with Canadian values (80% agree or somewhat agree)

Would the results have been different if the question specified “government” was “the arm’s length regulator, the CRTC?” Maybe, maybe not.

But what the second and third questions inadvertently demonstrate is that the many calls to change CanCon certification to a subjective “Canadian” test may provoke a negative public reaction, which is exactly what defenders of the current ‘nationality of creator” point system say, that the current certification system for Canadian content avoids the thorny issue of state-regulated cultural content.

***

And finally the poll’s headliner question, the one calculated to put a stake through the heart of regulating user generated content:

As you may have heard, Bill C-11, the Online Streaming Act, has passed in the House of Commons and is now pending Senate approval. The Act will extend the powers of the CRTC to regulate online streaming content. This will include regulating Canadian content, including how playlists and recommendations are provided to users.

How concerned are you that these changes will erode the streaming services Canadians can access?

The question makes no distinction between policing content (out of bounds in C-11) and asking YouTube to promote it, let alone how aggressively Canadian programs would be promoted and what actual interference there would be on recommendations relevant to search inquires.

***

Was this unpublished poll buried by YouTube after commissioning and paying for it? We don’t know why it was kept so quiet.

Still the question lingers: after two years of opponents publicly describing C-11 as the kind of ‘censorship’ bill you might find in North Korea, is it possible to solicit public opinion based on questions that aren’t calibrated to what is actually in the Act, or what is practically likely in its application?

Perhaps it’s all moot. The latest government messaging leans towards exempting YouTuber programs from discoverability regulation (mostly a good thing in the estimation of MediaPolicy.ca). Stay tuned.

***

If you would like regular notifications of future posts from MediaPolicy.ca you can follow this site by signing up under the Follow button in the bottom right corner of the home page; 

or e-mail howard.law@bell.net to be added to the weekly update; 

or follow @howardalaw on Twitter.