Challenges in using mixed methods to evaluate the performance of digital innovations in mission-driven organizations

A challenge for mission-driven organizations is to assess whether digital innovations actually support their goals. Indeed, and in contrast with for-profit organizations, mission-driven organizations have multiple and complex goals, which may be difficult to appropriately grasp. In this case study, we present a research design that used multiple methods to identify how a digital platform—the Congreso Virtual in Chile—performed inside a mission-driven organization, and whether the outcomes associated with the operation of the platform align with the organization’s mission. The research method entails the realization of three consecutive steps. First, we operationalized the organization’s mission and built a baseline from which we were able to evaluate the performance of the digital innovation. Second, we studied performance with three complementary research methods, namely, descriptive statistics; interviews; and a digital ethnography. Third, and last, we triangulated the results from the above methods to identify coherent findings across them, strengthening the validity of our results. The case shows the main challenges associated with each method, individually and collectively, and lessons regarding their application to this type of organizational context. We conclude by suggesting a few recommendations in terms of organizational politics and ethics when investigating and collaborating with real-world organizations.