Aktuelles

Data privacy and ethics in financial literacy evaluation research

Theresa Lorenz and Katharina Felbermayr 1

Like all research involving individuals, evaluation research presents specific ethical and data protection challenges that must be considered throughout the entire evaluation process, from the evaluation design to the final publication. In this introductory article, we provide an overview of the key legal and ethical aspects that are typically relevant for the evaluation of financial literacy interventions. To do so, we present important legal terminology and regulations alongside fundamental ethical principles. Furthermore, we suggest potential strategies for ­addressing ethical and data protection challenges.

Since evaluations of financial literacy interventions typically involve individuals, researchers must meet certain legal and scientific requirements. In the EU, the legal requirements primarily pertain to data protection, governed by both national and supranational laws, with the General Data Protection Regulation (GDPR) serving as the cornerstone. In contrast, in the US, there is no single equivalent to the GDPR. Instead, data protection is addressed through a sectoral approach and state-level regulations. Globally, countries maintain their own data protection frameworks, and even within the EU, member states incorporate additional national provisions, including specific derogations or margins of appreciation allowed under the GDPR. 2 Given the complexity and diversity of these legal frameworks, this article focuses primarily on the EU’s GDPR while referencing relevant national provisions of Austria where applicable.

The scientific requirements mirror the central principles of research ethics (e.g. the principle of voluntary participation), which should guide researchers’ ­actions. While the GDPR is grounded in ethical principles such as privacy and transparency, it transforms these principles into binding legal obligations that ­researchers have to meet. Therefore, data protection and research ethics overlap, but are not necessarily identical. For example, even if a researcher’s actions are adequate from a data protection perspective, they may still be questionable from a research ethics perspective (Unger, 2018). In contrast to data privacy laws, the field of research ethics is less standardized, with guidelines not being universally established at the national or supranational level. Instead, ethical standards – outlined in codes of ethics – can vary significantly. This variation arises because ethical considerations differ greatly based on who is being researched, the methods used and the specific research contexts.

In the following article, we provide an overview of data protection law and ­research ethics principles that are relevant for researchers working on financial literacy evaluation projects. The laws and principles presented are not exclusive to financial literacy evaluation research but rather apply broadly, to all research. Our contribution is adapting these general ethical standards and legal obligations to the context of financial literacy evaluation, highlighting the aspects most pertinent to evaluators in this field. Furthermore, ethical and legal considerations often receive limited attention in research in general (Felbermayr, 2023) and in existing financial literacy evaluation handbooks in particular.

This paper is structured as follows: Section 1 examines key aspects of GDPR compliance, addressing two crucial legal questions: When does the GDPR apply and what actions are required to ensure compliance? Section 2 explores core ­research ethics principles, highlights ethical challenges specific to financial literacy evaluations and offers strategies to address these challenges effectively.

1 Data privacy and the General Data Protection Regulation

In the context of evaluation research, it is often necessary to collect personal data. That means researchers need to pay special attention to both data protection regulations and ethical considerations when carrying out evaluations. To ensure adherence to data protection laws, particularly the GDPR, it is advisable to take the following steps before and throughout the evaluation process:

  • Understand the fundamentals of the GDPR: The EU’s GDPR is laid out in Regulation (EU) 2016/679 of the European Parliament and of the Council of the European Union. An essential first step involves getting familiar with key legal terminology as well as key principles and obligations for researchers. A concise overview of GDPR fundamentals will be provided below.
  • Conduct a data mapping exercise: For researchers, it is critical to document the types of personal data they are planning to collect, the purposes they will use them for, the data recipients (if any), storage locations and retention periods, and anyone who will have access to these data (Article 30 GDPR). This mapping ­exercise is vital for identifying potential compliance issues early on. It is also helpful for planning the technical and organizational safeguards necessary to protect personal data effectively and in compliance with the GDPR.
  • Consult data protection officers or GDPR experts: Given the complexity of data protection regulations, especially in projects involving multiple institutions or sensitive data, it is advisable to seek expertise from data protection officers (DPOs) or experts in data protection law. Many institutions offer such legal expertise. ­Expert guidance can be invaluable in navigating the nuances of GDPR compliance and of national law, as well as in implementing appropriate safeguards. ­Finally, if there is a DPO, they must be consulted (Article 38 para 1 GDPR).
  • Incorporate data protection by design and by default: From the outset of your project, integrate GDPR principles and compliance with data subjects’ rights (see below) into the planning and execution phases. The GDPR emphasizes the ­accountability principle, requiring organizations to take a proactive stance on data protection. This means designing your research project in a way that privacy and data protection measures are embedded at every stage, ensuring compliance as a fundamental aspect of an evaluation project.
  • Assess the risk for data subjects: Certain types of data processing, particularly those that rely on new technologies, are likely to result in a high risk to the rights of the data subjects. If a research project involves such data processing methods, a data protection impact assessment (DPIA) is required prior to processing. This assessment comprises a detailed analysis of all the risks and threats that could lead to a breach of any obligation laid down by the GDPR.

1.1 GDPR basics

The GDPR aims to protect personal data and applies to both private and public entities in the EU that process personal data. Personal data are any pieces of information relating to an identified or identifiable natural person. From this follows that data that does not qualify as personal data does not fall within the scope of the GDPR (i.e. anonymous or anonymized data or data relating to legal persons). In the table below, we define key GDPR terminology (Articles 4 to 10 GDPR; Mondschein and Monda, 2019; Wilms, 2019).

Table 1: Key GDPR Terms  
Term Definition
Processing Any operation performed on personal data, including accessing, collecting, saving, changing, analyzing, using, sharing,
­deleting, publishing, archiving data for follow-up research, etc.
Secondary Processing Processing of personal data that were originally collected for a different purpose, such as data from another research
­project.
Personal data Any information related to an identified or identifiable natural person (individual), also called data subject
Identifiable Person A person who can be identified with a serious possibility, either directly through direct identifiers or
indirectly through ­indirect identifiers.
Direct identifiers The International Organization for Standardization (ISO) interprets direct identifiers as
“data that can be used to identify a person without additional information or with cross-linking through
other information that is in the public domain” (ISO 25237, 2017, section 3.21). Information in the
public domain includes information that can be easily found on the ­internet. Examples for direct identifiers
are typically names, addresses, social security numbers, personalized email ­addresses or
phone numbers (Polonetsky et al., 2016).
Indirect identifiers Indirect identifiers are data points that can reveal an individual’s identity when combined with other information.
These typically include sociodemographic details like date of birth, age, gender, income, geographic location
(e.g. postal codes, ­census areas) or other characteristics that, when aggregated, may lead to the identification
of a person (Polonetsky et al., 2016). However, this does not necessarily imply that variables such as income,
age and gender always function as indirect identifiers. A person’s identifiability depends on the specific
context and the size of the population involved in the ­evaluation project.
Sensitive personal data Sensitive personal data involve a greater level of risk to the data subjects and thus require an additional legal basis
(Article 9 GDPR) and supplemental privacy measures if processed (e.g. a data protection impact assessment).
Sensitive data categories include racial or ethnic origin, political opinions, religious or philosophical beliefs, trade union
membership, genetic data, biometric data for the purpose of uniquely identifying a person, health data and information
about a person’s sex life or sexual orientation, among others.
Source: Articles 4 to 10 GDPR; Mondschein and Monda,2019; Wilms, 2019.

1.2 When does the GDPR apply?

The distinction between pseudonymous and anonymous data is decisive for whether the GDPR applies. While pseudonymous data still fall within the scope of the GDPR, anonymous data do not. However, even when a research project exclusively involves anonymous data, researchers must still adhere to established principles of research ethics (for more details on research ethics, see section 2).

Figure 1: Hierarchy of personal data types under the GDPR

The terms pseudonymous data and anonymous data can be summarized as ­follows:

  • Pseudonymous data: Data are defined as pseudonymous if all information that can identify an individual is removed (de-identification), but the individual can be re-identified with the help of additional information. This additional information is often referred to as a key. The GDPR still applies to pseudonymous data because re-identification remains possible as long as the key is accessible. Therefore, when handling pseudonymous data researchers must comply with the GDPR but may be subject to more lenient conditions due to reduced identifiability.
  • Anonymous data: Data are defined as anonymous if de-identification makes re-identification impossible, i.e. if it is impossible to link the data back to an individual, either directly or indirectly. Anonymization can occur after data collection (e.g. if the key to the data subject is destroyed). However, until the anonymization process is complete, the information is treated as personal data under the GDPR (Wilms, 2019).

Achieving true anonymization, particularly in evaluation research with individuals, is challenging and sometimes not even desirable. “[P]seudonymization under the GDPR is considerably far-ranging and circumventing compliance obligations under the GDPR by virtue of utilizing anonymous data is rather unlikely, as the usefulness of data for research purposes stands in contrast to the stringent criteria of anonymisation under the GDPR” (Mondschein and Monda, 2019, p. 60).

For example, to achieve anonymization, direct identifiers must be removed, and indirect identifiers must either be removed or generalized to prevent identification of specific individuals. However, in many cases, indirect identifiers like income, age or gender may contain critical information that is necessary for evaluation ­research (Polonetsky et al., 2016). We discuss possible approaches to addressing this challenge from an ethical standpoint in section 2.3.

1.3 What to do when the GDPR applies?

If the GDPR applies, researchers need to establish a legal basis for data processing (Article 6, 9 or 10 GDPR) and follow the principles of processing (Article 5 GDPR). Both steps will be described below.

Establish a legal basis for data processing: If the GDPR applies, one of the six legal bases defined in Article 6 GDPR for processing personal data must be fulfilled. In financial literacy research contexts, obtaining consent from participants tends to be the most relevant legal basis. This involves obtaining consent from individuals before processing personal data relating to them. Their consent must be (1) freely given, (2) specific, (3) informed, (4) unambiguous (Article 7 GDPR) and (5) the age of consent must be fulfilled Article 8 GDPR). Below, we provide further details on the criteria for consent.

(1) Freely given: Participants must have a genuine choice and be able to refuse or withdraw consent without facing negative consequences. This implies providing them with enough information to understand risks and alternative choices.

(2) Specific: Researchers must clearly articulate the purposes of data processing.

(3) Informed: Participants must be informed about the purposes and means of processing they are consenting to. This includes providing them with information on the right to withdraw their consent at any time and without any justification and negative consequences. Participants must also be made aware that, if they withdraw consent after data collection, their data will be deleted. Furthermore, they have a right to get information about the categories and the nature of the data being processed (e.g. sensitive personal data), details on who processes them (data controller) and on any third party the data will be shared with (data recipients, if any). Finally, researchers need to ensure that participants know about the data retention period and about their rights as data subjects (Chapter III GDPR).

(4) Unambiguous: Participants must give their consent through clear, affirmative action, such as signing a consent declaration, ticking a box or clicking a button. Silence, pre-ticked boxes, or inactivity cannot count as consent. The absence of negative consequences for participants withholding consent must be communicated.

The GDPR allows member states to set the minimum age of consent for data processing ­between 13 and 16 years (Article 8 GDPR).

Addressing the needs of children and vulnerable individuals: While not legally mandatory, ethically, it is advised to also seek and obtain consent from individuals with a legal guardian and children participating in research. This is based on the idea that “[c]hildren who are capable of forming their own views should be granted the right to express their views freely in all matters affecting them, commensurate with their age and maturity” (Wilms, 2019, p. 16).

Oral consent: The GDPR places a strong emphasis on the need to be able to demonstrate that the data subject has given their consent (principle of accountability). Thus, if oral consent rather than written consent is collected this must be documented somehow (e.g. recordings) (Wilms, 2019).

Evaluation research in schools and other institutional settings

When conducting evaluation research in schools, it is advisable to obtain approval for the study from both the school administration and the relevant educational authorities (e.g. ministry of education, local authority in charge of education, school principals, etc.). This step is also ­recommended when teachers are surveyed in their professional capacity – even if this happens outside of school hours. In some cases, national regulations may mandate such approvals as a legal requirement. However, legal obligations for research in institutional settings can vary significantly. For instance, it may be necessary to additionally obtain parental consent for students’ participation, even if the children have reached the minimum age of consent. Therefore, it is prudent to familiarize oneself with the specific legal requirements applicable to research in schools or any other institutional setting in question; or to seek legal advice before starting the evaluation.

Follow the principles of processing: When processing personal data, researchers must stick to the following GDPR principles: (1) lawfulness, fairness and transparency, (2) purpose limitation, (3) data minimization, (4) accuracy, (5) storage limitation, (6) integrity and confidentiality as well as (7) accountability. Below, we ­provide further details about the principles of processing.

(1) Lawfulness, fairness and transparency: Individuals must be informed in detail about the processing of their personal data, including the purposes, legal basis and rights available to them. Article 13 and Article 14 GDPR include a list of information that the data subjects need to receive before data relating to them are processed.

(2) Purpose limitation: Clearly define and communicate the purposes for which personal data are collected and processed to participants; ensure that their personal data are not used for any other purposes without a further legal basis (e.g. additional consent).

(3) Data minimization: Collect and process only the personal data that are necessary for the intended purposes. Avoid collecting excessive or irrelevant data. If anonymous or pseudonymous data are sufficient for the research purposes, such data must be used.

(4) Accuracy: Implement mechanisms to ensure the accuracy and currency of personal data throughout their lifecycle. Provide individuals with the opportunity to rectify inaccurate data.

(5) Storage limitation: Specify the period for which the data are needed to achieve the previously clearly defined purpose (retention period). Data must either be deleted or anonymized as soon as they no longer serve the purposes for which they were gathered. If the personal data are still needed for definite future scientific analysis, once again, this requires a specific legal basis (for further details, see the information on legal bases above). In the context of research and statistics in the public interest, Article 89 GDPR provides certain exemptions from this principle (for further details, see the information below).

(6) Integrity and confidentiality: In order to comply with the GDPR, it is advisable to follow the principle “data protection by design and by default” (Article 25 GDPR). Proactively implement appropriate technical and organizational measures to comply with all the GDPR principles, to uphold data subjects’ rights and to protect personal data against unauthorised access, disclosure, alteration or destruction. This includes having adequate data management plans as well as access rules for researchers and user authentication. Special measures are required if cloud computing services are used to store or process data, to safe electronic communication and for the use of certain other IT control instruments (e.g. backups). Pseudonymization techniques may also be necessary. More details on how to comply with confidentiality have been summarised by Wilms (2019).

(7) Accountability: As outlined in Article 5 para 2 GDPR, the researcher(s) or the organization they work for are responsible for ensuring and demonstrating compliance with all GDPR provisions. Therefore, if researchers claim that they have obtained consent, they need to be able to prove it.

Exemptions and safeguards: There are some exemptions and safeguards concerning personal data processed in the context of research and statistics in the public interest. These are outlined in Article 89 GDPR and for Austria additionally in: Section 7 Data Protection Act (Datenschutzgesetz – DSG) and the Research Organization Act (Forschungsorganisationsgesetz – FOG). Exemptions require appropriate safeguards for the rights and freedoms of data subjects, which may include data minimization, technical and organizational measures, privacy by design and by default as well as pseudonymization. In line with these principles, it is crucial to maintain comprehensive records of processing activities, including the nature, purposes and duration of data processing. Only then will the GDPR’s requirement for documentation and demonstrating compliance be fulfilled (Article 30 GDPR).

Applying pseudonymization and data segregation to comply with integrity and confidentiality

It is advisable to implement a pseudonymization strategy to demonstrate increased compliance with GDPR. Researchers can enhance data privacy by segregating identifiable personal information from other, non-personal data and creating two discrete datasets – one for identifiers and one for analysis. According to an advanced approach to data governance, researchers or the institution they work for should store, analyze and transmit these datasets independently (=institutional separation).

If, for example a research institution evaluates a program, the entity responsible for the program rollout may retain a dataset with identifiers while passing another, pseudonymous dataset on to the research institution in charge of evaluating the program. Spreading datasets across different institutions significantly reduces the risk of both datasets being compromised.

Table 2: Raw dataset  
Name Student ID Date of birth Grade School Attendance
%
Müller 987-654-001 April 12, 2005 8.2 Gymnasium Einstein 95
Luca Rossi 987-654-002 August 22, 2004 7.5 Istituto Leonardo 98
Source: Authors’ compilation according to best practices for ensuring data security in research in O’Toole et al. (2018).
Table 3: Identifiers dataset  
Name Student ID
Alice Müller 987-654-001
Luca Rossi 987-654-002
Source: Authors’ compilation according to best practices for ensuring data security in research in O’Toole et al. (2018).
Table 4: Pseudonymous dataset for analysis  
Student ID Grade School Attendance
%
987-654-001 8.2 Gymnasium Einstein 95
987-654-002 7.5 Instituto Leonardo 98
Source: Authors’ compilation according to best practices for ensuring data security in research in O’Toole et al. (2018).

Through this introduction to GDPR, we aim to provide financial literacy evaluation researchers with a foundation for appropriately protecting personal information. However, focusing solely on data privacy law will likely be insufficient. Researchers are also expected to adhere to ethical standards in their work. In the following section, we outline what constitutes ethical behavior in the context of social science research, where the relationship between researchers and participants is fundamental. Once again, we emphasize ethical considerations that are particularly important for the evaluation of financial literacy initiatives.

2 Research ethics

Research ethics are a central component of all research. In many disciplines, the ethical requirements researchers have to meet are summarized in a code of ethics. One example are the Ethical Guidelines for Educational Research 3 published by the British Educational Research Association (BERA, 2011). A code of ethics comprises the ethical principles and rules of a discipline in the form of recommendations. This also has to do with the fact that research methods and contexts can vary greatly, which, in turn, makes it impossible to implement a uniform set of ethical standards. It is the researchers’ responsibility to adapt research ethics guidelines to their specific needs and the study in question (Unger, 2018). Compared to interviews with adults, for example, research involving children gives rise to different ethical questions and challenges (e.g. when contacting them or obtaining their ­informed consent). The Compass for Research Ethics 4 provides a helpful overview when engaging in research with children and adolescents.

The intensity of the research ethics discourse varies from one country to ­another. Accordingly, the ethical requirements researchers must meet also vary strongly. Contrary to the USA or Canada, for example, in Austria or Germany research proposals in the social sciences do not have to be reviewed and approved by an ethics committee (Felbermayr, 2023; Kremsner, 2017). Leaving legal obligations to undergo a review aside, we think it is part of every researcher’s duties to reflect on their actions with a view to the central principles of research ethics.

2.1 What are the central principles of research ethics?

The following principles of research ethics should guide researchers’ actions ­(Felbermayr, 2023; Griffin and Balandin, 2004; Hatry et al., 2015; Unger, 2018):

  • Voluntary participation: Participation in evaluation research must be voluntary. Participants must be able to withdraw, or take a break, from participation at any point without fear of negative consequences.
  • Anonymity and confidentiality: Researchers must anonymize personal data by changing or removing all information that would allow others to link any participant to their data. In addition, researchers must maintain absolute confidentiality. This duty of confidentiality continues to apply even after the end of a research project. Personal data must never be passed on without the express consent of participants.
  • Potential for risk and harm: Participation in evaluation research must not lead to any harm to, or negative consequences for, the participants or their environment. When conducting e.g. qualitative, narrative interviews, there is a risk of retraumatizing participants when asking them to talk about past traumatic experiences. Researchers must assess all potential (long-term) consequences as best as they can. It might prove impossible, however, to realistically anticipate all risks of harm, such as harm to participants’ mental health. If necessary, researchers should refer participants on to contact points offering professional support.
  • Transcripts provided by third parties and publication: Researchers must inform participants about what happens to their data. This applies, for example, when reproducing anonymized quotations in scientific publications or when contracting third parties (e.g. external agencies) to transcribe audio files. Informed consent means a participant’s verbal or written consent to participate in evaluation research. Written consent means having a consent form that is signed by the participant and that contains essential information about research ethics and applicable data protection law (for further details on data privacy, see also section 1.3).

“Obtaining active, written consent is the most transparent way of holding an evaluation accountable. But an alternative is passive, verbal consent. In this format, research administrators would develop a description of the research effort that includes all the elements from the above bullets and would read it to potential participants exactly as written so that there is no variation depending on which member(s) of the research staff may be recruiting the participant. Once research administrators have finished reading the script, participants may then ask any questions they might have and are given the opportunity to verbally refuse participation, without a consent form. In some cases, verbal consent is more appropriate for literacy-related reasons: Those who have low literacy may not be able to read or fully understand the consent form, nor may they be able to meaningfully provide written consent” (Yoong et al., 2013, p. 204).

Participants can thus give verbal or written consent, but their decision to consent always has to be an informed one. To be able to make an informed decision, potential participants must receive detailed information in advance (Lewis and Porter, 2004). However, there is no consensus in research on how much information participants should receive. At one end of the spectrum, it is considered necessary to fully inform participants about the project, including its research questions, whereas, at the other end, it is considered sufficient to communicate some general aspects of the project (Kruse, 2015). In the end, it is up to each research team to decide how much information they wish to share and to be transparent about this decision in their research report. In this context, we would like to mention an evaluation study we conducted on the implementation of new financial literacy interventions in kindergartens. Before the start of the study, we provided participants (teachers, parents) with information materials and spoke to them about the background of the study, the research design and the planned surveys. Providing this kind of information meant fully educating participants about the project, which has proven to be a good basis for a trusting relationship with them.

Note

Obtaining informed consent for conducting a study can be a lengthy and time-consuming process. When evaluating e.g. financial literacy interventions in institutional settings, the school principals and authorities in charge may also have to give their consent. “Evaluators may have to consider alternatives such as requesting data without any personal identifiers or requesting group data only” (Hatry et al., 2015, p. 819).

2.2 What are the ethical challenges involved in qualitative and quantitative evaluations?

Challenges regarding research ethics are as varied as the research settings and the research methods applied. In the context of evaluations, ethical challenges arise more often than in other research contexts “due to the applied and political nature of the work, along with the existence of diverse sources of guidance that do not suggest a single ‘correct’ response to a given ethical dilemma” (Hatry et al., 2015, p. 819). Particularly when it comes to qualitative research, questions concerning research ethics tend to be difficult to answer, given the diversity of qualitative ­research approaches and individual research questions (Hopf, 2012; Unger, 2018). Quantitative evaluation research, by contrast, may come with fewer ethical challenges due to often limited personal contact with participants. However, this, too, depends entirely on the respective research setting and design. The following list contains a number of ethical challenges that may arise in both qualitative and quantitative research.

  • Conflicts of interest: When conducting evaluation studies, the various parties ­involved (e.g. funding bodies, implementers, researchers, political agents) may enter into conflict. Maintaining objectivity and independence can be challenging, for example, when researchers are supposed to independently evaluate a ­financial literacy intervention while, at the same time, being paid by the body funding the intervention (Yoong et al., 2013, p. 210). Yoong et al. (2013) recommend the following basic rules of cooperation.

“Specifying roles of funders and the implementing agency in the evaluation. Establishing clear lines of communication between evaluators, implementers, funders, and other relevant stakeholders. Articulating and committing to the evaluation’s standards for transparency and accountability. Describing any known or potential conflicts of interest” (ibid., pp. 210-211).

  • Ensuring complete anonymity: In the context of evaluations, collecting personal data is often unavoidable. However, a key principle of research ethics is to guarantee anonymity during data analysis and publication. This means that researchers must remove or change any information that could lead to an individual being identified (e.g. name, region, institution, profession) in qualitative interview transcripts and publications. Doing so comes with certain challenges for qualitative studies: On the one hand, researchers must ensure the promised anonymity by removing all personal identifiers from interviews. On the other hand, they must not reduce the information contained in the interviews to such an extent that the specific context of an interview is lost (Felbermayr, 2023). It is debatable whether complete anonymity can ever be obtained, especially in this day and age, where the internet allows us to identify participants based on a combination of known personality traits (Lewis and Porter, 2004; Unger, 2018). The way a person speaks or the (technical) terms they use may also allow us to draw conclusions about them. Despite such difficulties, researchers must not forgo anonymity. They must take promises of anonymity and confidentiality seriously and protect participants as best as they can. When addressing anonymity, researchers must also comply with all applicable data protection law. In line with this, researchers must keep personal data only for as long as is necessary for the purposes of a given research project. They must also have an appropriate legal basis for processing personal data, such as informed consent (for further details, see section 1).
  • Benefit or added value for participants: In the context of research, participants’ ­motives for, and benefits of, participating in studies also play an important role. Imbalances may arise in particular for research conducted to obtain an academic degree. In these cases, the added value for the researchers (e.g. obtaining an ­academic title) is higher than for the participants. It is therefore important for researchers to think about the added value for participants and how to thank them for their participation. This also raises the question of whether or not ­incentives should be offered to participants for participation. The academic ­debate about monetary incentives is controversial. According to some, a greater willingness to participate would speak in favor of such incentives. Others claim, however, that participants should not take part in research for monetary reasons, but primarily out of personal interest.
  • Ensuring informed consent: Another important question is how to ensure that ­potential participants in a research project understand the project-related information they receive in order to be able to give their informed consent. This question is particularly relevant for research involving children, people with special needs or people with low literacy. “Whether informed consent is written or oral, it is critical to realize that in dealing with people with low literacy, even basic financial terms – such as ‘interest’ or ‘budget’ may not be understood and should be defined in the consent process. It is well-documented that access to financial services in the developing world is extremely low, so basic financial terms are likely to be unknown to many participants” (Yoong et al., 2013, p. 204). The use of supporting materials (such as pictures) can minimize the risk of people agreeing without understanding the information provided (Griffin and Balandin, 2004). Documents in plain language can also be helpful, as they present complex content in a simplified form.
  • Research involving vulnerable groups: Researchers need to be particularly considerate when working with members of certain groups, namely “vulnerable and underserved populations – such as the poor, children, the elderly, those with little education, and those with mental or physical illness or disabilities [...]. [...] These populations may also be more likely to experience negative repercussions from participating in certain types of research and evaluation, including risks to their personal safety, social ostracism, or exclusion from a particular program” (Yoong et al., 2013, p. 201).
  • Building relationships: Qualitative research in particular relies on research methods that usually allow researchers to enter into direct contact with participants (e.g. through interviews or focus groups). This gives rise to questions about the nature of the relationship between researchers and participants, which is less of an issue in quantitative research. As researchers and participants usually do not know each other before the start of a project, establishing a trusting working relationship is essential for the success of (qualitative) research. The challenge for researchers lies in maintaining a good relationship with those involved in their projects, while also making sure that participants do not confuse this relationship with friendship. This is particularly true for longitudinal studies, where the working relationship with participants may grow over time, given their participation in several surveys (Detamore, 2010; Felbermayr, 2023; Thomson and Holland, 2003).
  • Discussing sensitive topics: Qualitative research in particular may also involve (planned or spontaneous) personal conversations about sensitive topics. This may lead to the retraumatization of participants and may also cause psychological stress for them (Kelle and Erzberger, 2006). Researchers are responsible for those who participate in their projects and must not leave participants in a state of agitation following an interview. In social research in particular, it is advisable to prepare a list with the contact details of psychosocial support services and to hand out this list during or after interviews, if necessary (Felbermayr, 2023). Topics related to finance, such as debt, may also be a sensitive subject area for some people. Researchers conducting evaluation research should be aware of this, too.

In this section, we provide an overview of relevant ethical questions that researchers should consider for evaluation research. We hope to have clarified that – contrary to legal regulations – the standards of research ethics can only ever serve as a point of orientation for researchers’ actions. It is every researcher’s responsibility to take, and be transparent about, their own decisions concerning research ethics. These decisions, of course, will always depend on the context of their research (research method, target group, etc.).

Summary and concluding remarks

In this paper, we outlined key data privacy and ethical considerations for evaluation researchers given that processing personal data is often inevitable in evaluations. We discussed essential concepts from the EU’s General Data Protection Regulation (GDPR) that are necessary for understanding the legal framework. Moreover, we provided fundamental legal guidelines to ensure GDPR compliance.

While formal legal frameworks like the GDPR mandate accountability in data processing, research ethics offer broader principles to guide responsible research. Ethical concerns often overlap with data privacy issues, as both frameworks emphasize the protection of participants’ personal information. This includes obtaining their informed consent and ensuring that personal data are used only as long as necessary. At the same time, decisions regarding data privacy and research ethics are often complex, requiring careful judgment and the weighing of multiple factors.

For example, financial literacy interventions often target children and adolescents (Mauser et al., 2024). This raises significant research interest in conducting evaluation research with children and young people. Ethically, children and young people are considered “social actors in their own right and should have a voice in research” (Vogl et al., 2023, p. 2). At the same time, research involving vulnerable groups, such as children, demands stricter data protection measures and ethical considerations. Researchers must work closely with data privacy and ethics experts to carefully balance their objectives with evolving privacy regulations and ethical standards. It is important to say that this should not discourage researchers from conducting evaluations involving vulnerable but highly relevant groups.

Examples like this only offer a small glimpse into the challenges and considerations that are involved in data privacy and research ethics. Since this article can only be an introduction to data privacy and research ethics we highly recommend consulting experts in both areas. In the context of the GDPR many evaluation projects may present specific challenges, with national variations and continuous updates of the regulation going far beyond the scope of this article. Moreover, if a research institution has appointed a data protection officer (DPO), it is mandatory to involve the DPO in any evaluation project concerning data privacy matters. The contexts of and the resulting ethical challenges in both qualitative and quantitative evaluations are diverse. Therefore, we also recommend engaging with the ethical stakes involved in such research beyond this introductory article. Researchers are encouraged to reflect on potential ethical challenges with their research fellows and delve deeper into the relevant literature to gain a more comprehensive understanding of these complex issues.

References

BERA. 2011. Ethical guidelines for educational research. https://www.bera.ac.uk/publication/ethical-guidelines-for-educational-research-fifth-edition-2024

Detamore, M. 2010. Queer(y)ing the ethics of research methods: Towards a politics of intimacy in researcher/researched-relations. In: Browne, K. and C. J. Nash (eds.). Queer methods and methodologies. Intersectioning queer theories and social science research. London: Routledge. 167–182.

The European Parliament and the Council of the European Union. 2016. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official ­Journal of the European Union. L 119/1–88.

Felbermayr, K. 2023. Entscheidungsprozesse am inklusiven Übergang. Eine Grounded Theory Studie im Längsschnitt. Verlag Julius Klinkhardt. https://doi.org/10.25656/01:26629

Griffin, T. and S. Balandin, S. 2004. Ethical research involving young people with intellectual disabilities. In: Emerson, E., C. Hatton, T. Thompson and T. R. Parmenter (eds.). The international handbook of applied research in intellectual disabilities. Chichester: John Wiley & Sons Ltd. 61–82.

Hatry, H. P., K. E. Newcomer and J. S. Wholey. 2015. Evaluation challenges, issues, and trends. In: K. E. Newcomer, H. P. Hatry and S. J. Wholey (eds.). Handbook of practical program evaluation. Hoboken: John Wiley & Sons, Inc. 816–832.

Hopf, C. 2012. Forschungsethik und qualitative Forschung. In: Flick, U., E. v. Kardorff and I. Steinke (eds.). Qualitative Forschung. Reinbek bei Hamburg: Rowohlt Taschenbuch. 589–600.

ISO. 2017. Health informatics — Pseudonymization (ISO 25237:2017). International Organization for Standardization.

Kelle, U. and C. Erzberger. 2006. Stärken und Probleme qualitativer Evaluationsstudien. Ein empirisches Beispiel aus der Jugendhilfeforschung. In: Flick, U. (ed.). Qualitative Evaluationsforschung. Konzepte, Methoden, Umsetzungen. Reinbek bei Hamburg: Rowohlt Taschenbuch. 284–300.

Kruse, J. 2015. Qualitative Interviewforschung. Ein integrativer Ansatz. Weinheim: Beltz Juventa.

Lewis, A. and J. Porter. 2004. Interviewing children and young people with learning disabilities: Guidelines for researchers and multi-professional practice. In: British journal of learning disabilities 32(4). 191–197.

Mauser, M., V. Voith, M. Razen and S. Humer. 2024. Monitoring-Report der Nationalen Finanzbildungsstrategie für Österreich 2023. Nationale Finanzbildungsstrategie.

Mondschein, C. F. and C. Monda. 2019. The EU’s General Data Protection Regulation (GDPR) in a research context. In: Kubben, P., M. Dumontier and A. Dekker (eds.). Fundamentals of clinical data science. Cham: Springer. 55–71.

O’Toole, E., L. Feeney , K. Heard and R. Naimpally. 2018. Data security procedures for researchers. J-PAL North America. https://www.povertyactionlab.org/sites/default/files/Data_Security_Procedures_December.pdf

Polonetsky, J., O. Tene and K. Finch. 2016. Shades of gray: Seeing the full spectrum of practical data de-identification. In: Santa Clara law review 56(3). 593–629.

Thomson, R. and J. Holland. 2003. Hindsight, foresight and insight: The challenges of longitudinal qualitative research. In: International journal of social research methodology 6(3). 233–244.

Unger, H. v. 2014. Forschungsethik in der qualitativen Forschung. Grundsätze, Debatten und offene Fragen. In: Unger, H. v., P. Narimani und R. M´Bayo (eds.). Forschungsethik in der qualitativen Forschung Reflexivität, Perspektiven, Positionen. Wiesbaden: Springer. 15–39.

Unger, H. v. 2018. Forschungsethik, digitale Archivierung und biographische Interviews. In: Litz, H., M. Schiebel and E. Tuider (eds.). Handbuch Biographieforschung. Wiesbaden: Springer Fachmedien. 681–693.

Vogl, S., E. M. Schmidt and O. Kapella. 2023. Focus groups with children: Practicalities and methodological insights. In: Forum Qualitative Sozialforschung Forum: Qualitative Social ­Research 24(2). https://doi.org/10.17169/fqs-24.2.3971

Wilms, G. 2019. Guide on good data protection practice in research. European University Institute.

Yoong, J., K. Mihaly, S. Bauhoff, L. Rabinovich and A. Hung. 2013. A toolkit for the evaluation of financial capability programs in low-, and middle-income countries. World Bank.

1 Oesterreichische Nationalbank, Financial Literacy and Culture Division, theresa.lorenz@oenb.at,
katharina.felbermayr@oenb.at. Opinions expressed by the authors of studies do not necessarily reflect the official viewpoint of the Oesterreichische Nationalbank, the Bank of Greece or the Eurosystem. The authors express their gratitude to Bernhard Horn (OeNB) for valuable comments and suggestions. This paper is part of the OeNB ­Financial Literacy Evaluation Series. The series aims to inform researchers, policymakers and educators about the current state of research on financial literacy and education and to provide guidelines for designing and implementing comprehensive evaluation studies. The series was developed by the OeNB in collaboration with the Bank of Greece. For details and further publications of the series, see OeNB Financial Literacy Evaluation Series - ­Oesterreichische Nationalbank (OeNB) .

2 In Austria, for example, researchers also need to comply with the Austrian Data Protection Act ­(Datenschutzgesetz – DSG) and the Austrian Research Organization Act (Forschungsgesetz – FOG).

3 https://www.bera.ac.uk/publication/ethical-guidelines-for-educational-research-2011

4 https://core-evidence.eu/compass-for-research-ethics

Inhalt