Privacy practices in the Indian technology ecosystem

Both qualitative and quantitative methods were employed to conduct this research.

Sections:

[TOC]

Qualitative research

We finalized the broad hypotheses and research questions in April–May 2020, and set up interviews with experts in June–July 2020. Experts were shortlisted on the basis of business sector diversity, domain expertise and industry experience.

In the first round, six experts were interviewed:

1. A data security practice head of a large consumer-internet startup
2. An information security practitioner from a multinational services organization
3. A senior engineer working in a large social media organization
4. An independent machine learning (ML) consultant with 15+ years of experience working in startups and multinational corporations (MNCs)
5. Product head of a health startup
6. A practicing Supreme Court (SC) lawyer

Each interview was held via online conference call, and was recorded with the consent of participants. Interviews were transcribed for reference. A second round of expert interviews were held in October–November 2020 to develop a historical account of privacy, corporate governance and business imperatives globally and in India. Four interviews were conducted with:

1. The founder of a civil society organization looking at gender justice
2. The co-founder of a legal policy and advocacy organization
3. The founder of a legal-tech consultancy
4. The founder and CTO of a media platform startup

Composition of focus groups and identification of participants

Focus Group Discussions (FGD) were conducted to learn:

1. The top-level hypotheses of resource allocation and operational priorities when it comes to building privacy-enhancing technologies.

2. Agency of individuals, mainly their ability to influence and shape decisions about incorporating privacy features into products and about the governance of user data.

3. The participants’ personal views about privacy.

FGD groups were heterogeneous. Participants were shortlisted on the basis of experience (senior roles, tending towards management versus frontline execution), and with diversity in sector, gender, location and expertise. Six FGDs were conducted online, via video calls, with participants from paytech, fintech, SaaS, social networking, and health tech. FGDs were recorded and transcribed with the consent of the participants.

Hypotheses formulation and quantitative survey design

As the research progressed in June–August 2020, the high-level hypotheses were fine-tuned based on the learnings from qualitative research. The initial hypotheses from April–May 2020 required surveying middle management and junior employees to understand their skill and competence, and their ability to influence decisions for implementing privacy-enhancing technologies inside organizations. The qualitative interviews revealed that:

1. At the junior level, there is neither authority to influence change, nor competence, nor awareness. Their opinions do not reflect the organization’s intent or policy, and we will necessarily have to look at other ranks for information.

2. At the middle management level, there is no authority, but there can be competence and intent.

3. At the executive level, there is authority and some intent, but uncertain competency.

The findings from the qualitative research strongly suggested that the quantitative survey’s focus must be shifted to the executive and middle-senior management to understand intent, processes, including hiring for privacy and compliance, and questions about business imperatives to determine the level at which privacy concerns were considered to be part of the organization’s overall culture. The quantitative survey was developed based on this renewed research direction.

“I deal with data scientists day in and day out, they think of themselves as algorithmic engineers, there are some latest and greatest algorithms and my job is to apply it to a given problem and they are interested in the business matrix around it. The degree to which value and judgement is applied is very limited, I’m talking about some of the senior most people. I’m not even talking about junior people, I track some of these things, the kind of advanced conversations that happen elsewhere rarely happens here. So this has to do with the larger training, the larger incentive structures and so on.”

—A founder of a machine learning startup, during an interview

Survey design and revision

Our initial survey consisted of approximately 20 questions, including a preliminary section that asked for information about the respondent – organization name, designation, nature of business – and a section about the organization’s privacy practices. We sought this information to classify respondents into those with authority and those without, to rank organizations based on their sizes, and to ensure we had a more diverse, representative sample. This section also asked respondents for personal information: their name, level of education, city, gender and caste group. However, personal questions were marked optional and respondents could skip them. See our note on why caste is relevant.

We disclosed the survey’s sponsor upfront at all stages. Some initial respondents asked if survey data would be shared with the sponsor. To clarify this, a confidentiality and ethics statement was added, along with the research team’s contact details, in case respondents required further clarifications.

We took efforts to ensure the survey would not take longer than ten minutes to complete, to lower the risk of abandonment.

The research team revised the survey a total of four times, based on internal input and external feedback. We approached three experts – in fintech, law, and AI ethics respectively – to review the questionnaire on:

1. Length and time required to fill it
2. Choice of questions about respondent information
3. Clarity of explanations and context to the overall survey
4. Depth and validity of the information sought
5. Whether questions were framed appropriately to draw out sector-specific information

Their recommendations:

1. Explain the logic of questions upfront, especially when collecting sensitive data
2. Explain sector-specific background such as the mechanisms that regulated organizations follow to adhere to compliance
3. Set expectations for respondents, in terms of length of questionnaire and time required to fill it

We tweaked the survey questionnaire based on the feedback, with additional questions to detail out demographic information from respondents. We reached out to potential respondents – drawn from personal and professional networks – via email, as well as sharing on hasgeek.com and social media.

Bias, survey administration, and ethical undertaking

At all stages of the research process, we took steps to ensure that our own inherent biases do not reflect in the survey, and influence the respondents in any way. Both qualitative and quantitative parts of the research began with an ethical undertaking in which we disclosed organizations and supporters involved in the survey, and provided information regarding purpose of the research – what benefits or compensation respondents can expect, guarantee of anonymity, and publication of the findings.

Given the objective of the research was to learn about privacy practices, the survey was designed to elicit information about actions taken or in effect in organizations, rather than the attitudes and beliefs of respondents. Discussion points in the qualitative interviews and FGDs were drafted to understand specific practices and to confirm hypotheses, but the hypotheses themselves were not shared with participants. The questionnaire for the quantitative part of the survey was designed to elicit actual practices and not desired states. Respondents of the survey remained largely unknown to the research team and the survey was administered online, limiting the need for direct contact.

As mentioned in the section on Interviews and FGDs, and in the survey design section, respondents were chosen from both personal and professional networks, and the survey itself was shared on social media in order to get diverse participation. Participants for qualitative interviews and FGDs were shortlisted for not just their sectors and experience, but also to get a more representative sample from across the industry. Informed consent was obtained.

The Intent, People, Process (IPP) framework

We analysed the survey results using an Intent, People, Process (IPP) framework, which is based on the Technology, Process, People and Culture framework of the Capability Maturity Model (CMM) that is commonly used in organizations, especially in tech organizations, to understand and assess an organization’s internal dynamics. Improvements in capability are an outcome of improvements in these four areas.

Given that the survey was looking at how organizations approached privacy respecting features of the products and services they were building – that is, the technology – the framework needed to be altered. We chose to analyse an organization’s intent – the stated objectives and policies towards privacy, and the budgetary allocation.