DFI Measurement and Impact Report 2019

Foreword from Gavin Krugel

The importance of measurement and impact of initiatives and meeting customer needs are topics which feature in DFI’s courses. Over the last four years DFI tracked its own value as a capacity-building institution by monitoring and measuring impact. Our scope for measuring our impact is on the places and the people to whom we provide our courses. In addition to surveying students before and after training, 6,12, and 18 months post-completion, DFI commissioned third-party monitoring and evaluation exercises each year to measure impact on practitioners, organizations and even countries that take our courses.

What follows is a summary written by Dr Andrew Hartnack, from Development Works Changemakers, of the most recent (2019) assessment including five case students (two countries and three institutions involved in Digital Finance in Africa) and a Monitoring and Measurement of Results Report. The two country case studies are Zambia and Uganda and Accion, Bank of Kigali and MTN were selected for institution-level impact case studies. All of the studies reveal a common thread regarding the value that institutions and countries gain by partnering with DFI[2], consistent with prior years M&E exercises.

We are also pleased to say that Students have consistently rated DFI courses highly, citing improved knowledge about digital financial services; increased awareness of international examples and opportunities in the wider sector; increased confidence in their ability to use their knowledge to participate in conversations and apply what they’ve learnt inside workplaces or communities of practice.[1]

The full results report and case studies are also available for you to download and we encourage you to read at least one of the case studies to see what impact could be made in your organisation or country. We thank those students and organisations who completed the questionnaires and took part in interviews, and also Development Works Changemakers for compiling the reports.

 

1. Impact on Practitioners[3]

The report finds that the practitioners felt that their capacity had been significantly enhanced, something which both they as individuals and their colleagues or managers would be able to verify. Many felt that they had gained significant increases in DFS knowledge, confidence and that their minds had been opened, which had helped them to become more effective at work. Practitioners also felt that they had become more effective, and better able to contribute new ideas and innovations. The vast majority of practitioners felt that they were using the knowledge they gained from the training, many of these on a regular basis. A significant number indicated that they expect their new knowledge to allow them to make more contributions in their workplace, while a majority also stated that they are already using this knowledge to make important contributions.

Unsurprisingly, given the short time since completion of the training for both surveys, more than half of respondents reported that they had not changed their work situation since graduating. However, among those who had, many reported increased satisfaction as a result of a promotion or increased scope of their work and even increased remuneration.

Overall, the data from the 6 and 18-month follow-up surveys show that DFI has had a positive impact on the capacity of individual practitioners in several work-related areas and that this has acted as a catalyst for broader positive change in their workplaces, in their industries and potentially, in their countries.

The qualitative analysis backs up these findings and delves deeper into the extent to which each indicator has been addressed by DFI. For the indicator of increased DFS capacity among practitioners, the data shows that there has been a very tangible increase in DFS capacity as a result of their DFI training. DFI practitioners clearly have gained significantly in terms of knowledge, both general knowledge about the DFS landscape (which has been crucial to their effectiveness at work), and more specialised knowledge which has helped them in particular areas (e.g. payments, blockchain, anti-money laundering, regulation). The knowledge these practitioners have gained has helped many of them to add new skills in developing and managing products and projects in the DFS space, improved consulting with clients, lobbying regulators, and being more confident to develop innovations and collaborations.

Around half of these individuals have advanced their careers into new positions and responsibilities, which they attribute clearly to their DFI experience. Some are now driving new strategic directions for their companies as a result of their involvement with DFI. Lastly, around two-thirds of the practitioners also report that their capacity has also been enhanced through greater networks and networking ability, through not only the courses but also the Communities of Practice (CoPs) and other initiatives which have grown out of the CoPs. Individual capacity development through all these aspects is clearly a very strong impact emerging from the DFI courses.

In respect of the indicator relating to increased DFS service provider capacity, the following was found: Although harder to make a clear judgement on organisational capacity development as a result of the DFI courses, 74% (14 out of 19 practitioners) provide some evidence that their individual capacity enhancement has also translated into institutional capacity. Those working for commercial banks, in particular, have shown great capacity to assist their banks to realise their strategic goals of making their services digital in the coming years. They have done this through understanding the needs of customers more clearly and by developing a range of digital products and services which help their banks to offer DFS to their clients. Other practitioners working in the consulting field or in the non-profit sector have similarly added value to their organisations by leading them to become more effective DFS providers. At larger organisations, such as the Bank of Kigali, the fact that a significant number of key staff have taken the DFI courses has fed directly into the bank’s ability to enhance its capacity. They are therefore planning to invest in more employees taking DFI courses and thereby further enhancing their capacity.

On the increase of DFS service provider capacity for regulation, the following findings were made: Although it is difficult to make a conclusive evaluation of how individual practitioners increase their institutions capacity for DFS regulation, again, at least 74% (14 out of 19 practitioners) demonstrate some level of influence in this regard. In some cases, organisations were simply too large, with an already high capacity for such regulatory input, for one or two DFI alumni to make a real difference. But in other cases, DFI alumni were playing key roles which enabled them to contribute to their organisation’s regulatory capacity in some way. Consultants working in the DFS space felt that their DFS knowledge had enabled them to help clients with regulatory issues. Similarly, those working for non-profit organisations (often networks of members) also demonstrated increased capacity and understanding for DFS regulation which was helping their organisations and members. Even at large commercial banks, there was some evidence that individuals in key roles could impact the regulatory environment through sub-committees of their Bankers’ Associations. Individuals working for a regulator also felt that increased knowledge of the DFS field could help to guide regulations from within the Central Bank.

It was also found that almost 60% of DFI practitioners who were interviewed have been involved, in various capacities, in supporting or developing 30 different initiatives relating to regulation and policy of DFS. Some practitioners are working directly with government regulators (as consultants) to assist them to develop regulations in a range of areas. Others, such as those at the commercial banks, are inputting through committees of their country banking associations. A few have also been heavily involved in developing strategies and policies at the country level. A few organisations active in lobbying government are also actively organising national dialogues or preparing policy briefs which influence the regulatory environment. As is evident, this small sample of DFI practitioners is fairly active in attempts to influence and develop DFS regulation in sub-Saharan Africa and beyond. They have thus made an impressive contribution to the DFS policy landscape through these efforts.

Lastly, on the topic of cross-sectoral collaborations, it was found that while most practitioners felt their cross-sectoral networks had expanded and they could draw on these connections when necessary, 58% had achieved more concrete collaborations of various kinds. The consultants had experienced a widening of their client base and scope of work as a result of their DFI experience, while those working in the NGO space also felt they had more connections which were useful to them in their work. Similarly, individuals working for government entities collaborated with non-state actors and bodies in the course of their work, having found the DFI courses useful in helping them in these endeavours. Meanwhile, those working for commercial banks found that in developing new digital finance channels and products, it was crucial to collaborate with a wide range of sectors. Overall, the DFI courses did enable these practitioners to collaborate more broadly and more effectively, where previously they did not have as much confidence, or the network, to do so.

In addition to the above areas of investigation, it was also found that practitioners and other interviewees overwhelmingly supported DFI continuing its certification partnership with Tufts University. It was felt that this provides the courses with international credibility and a brand name, which many in Sub-Saharan Africa value greatly.

 

2. Impact on Organisations

The three institution case studies along with the two country cases studies (Zambia and Uganda) reflect common impacts across a wide range of Sub-Saharan African countries. These include the clear enhancement of capacity in practitioners who take the DFI courses, the creation of a common DFS language and understanding among colleagues, which helps them in their work and strategic thinking, and the new tools for collaboration and policy influence, such as the Communities of Practice and various online forums and associations which have been initiated as a result. Together, these have been contributing significantly to the growth and development of the DFS landscape in many countries in Africa.

The Case Study on Accion found that the DFI courses have clearly helped develop capacity and to raise the level of their contributions to the company, and more broadly to the DFS industry. Senior staff have taken the DFI courses and recognised them as a key tool in helping the organisation to bring its staff up to speed on the fast-changing landscape of DFS globally, citing the importance of having an accessible body of up-to-date industry knowledge through DFI to inform strategic direction and the areas they should be giving attention to in future.

Senior staff also pointed to the DFI course’s value in enhancing critical thinking among teams, which has enabled them to do their work with greater impact. Companies have also been helped significantly in implementing digital strategies by sending a large number of practitioners through DFI courses. Managers argued that fellow team members who took DFI courses were able to develop a common understanding and language of issues relating to DFS, which helped them considerably in their work. For example, Robert Magala was able to assist the Accion Microfinance Bank (Nigeria) to roll out a digital savings product due the knowledge he gained from the DFI course he took. Magala also used his new knowledge of DFS to assist the bank to rethink its microloans for traders product as a digital product, and the literature he was introduced to in the DFI courses helped him to design the programme and themes for an important multi-stakeholder conference on microfinance in Nigeria.

It is therefore not surprising that Accion International has paid for up to 50 personnel to take the DFI courses, and as illustrated above, senior managers have indicated that this has helped to build the staff’s knowledge and capacity. Moreover, the courses have helped these senior staff themselves to be up-to-date on the rapidly evolving DFS industry, and to make a number of key strategic decisions about the focus of Accion and its arms, such as the CFI. USA-based staff who do not directly work in the DFS space have also gained from taking the courses. For example, key fundraisers have found that the courses have allowed them to more effectively mobilise resources for Accion’s important projects, due to insights they received in the DFI courses. It is fair to conclude, therefore, that the DFI courses have been of great value to Accion International, and made a significant positive impact on its work in various areas, as well as in the institutions it supports in Sub-Saharan Africa.

The Case Study on MTN has shown that the DFI courses have had a positive impact upon all of the DFS practitioners working for MTN around sub-Saharan Africa. At an individual level, MTN employees felt that their knowledge of the DFS ecosystem, and theory around DFS had been improved, as well as their practical knowledge of what is happening in other countries. Many of them said they could apply this knowledge to their local context and everyday working life, which helped them greatly. Although it is not possible to know the extent to which DFI has impacted the MTN Group in Africa, and assisted it in realising its vision, subsidiaries in the countries covered certainly benefitted, especially where several employees took the courses. This was most evident in Benin, where all MTN Mobile Money employees are required to take the CIDM. This has created a critical mass of practitioners speaking the same language and in-tune with the same issues. It is certainly helping them to drive MTN’s role in Benin forward. Practitioners in the other countries were also positive that their role at a more senior level allowed them to apply their learnings at a more strategic and impactful level.

Examples of such impacts included practitioners in Benin developing an MTN Mobile Money savings and lending product, and an electricity bill payment product, having been helped through knowledge gained in DFI courses. Practitioners also noted knowing about the background of mobile money, which regulatory factors affect mobile money, and what is happening around the world in the field of mobile money as being key areas of insight the gained through DFI. Other products that were developed with the use of DFI knowledge include an e-commerce platform for mobile money in Benin (based on a model used in China), and new partnerships between MTN, banks and fintechs to improve e-cash accessibility through loans to MTN agents. In Zambia, a practitioner whose job involves liaising closely with regulators was assisted greatly through knowing a number of government officials through DFI networks. DFI has thus clearly contributed to MTN Mobile Money in these countries in various ways, as it likely also done in other countries in which MTN is operating.

The Case Study on Bank of Kigali, as one of the oldest and largest commercial banks in Rwanda, and one of Africa’s leading banks, has been undergoing a radical transformation in the last few years to become a leading digital bank. The case study has shown that DFI has played a major role in assisting the bank on this transition. Not only has the knowledge, skills and networks gained through DFI courses helped individual bank staff to develop their capacity, and to do their jobs better. But the courses have certainly had an impact in certain key units, such as the Digital Factory, the Transformation unit and others. Examples of impact include the fact that senior managers feel that their staff are now able to communicate using the same language and strategise and problem solve due to their common understanding of DFS. It was pointed out that as a market leader, BK needed a large pool of practitioners who could help drive the vision forward and make an impact even beyond the bank. Senior managers in the bank have also encouraged female employees in particular to take DFI courses, to ensure that the skewed gender profile of the DFS industry can be addressed. BK’s Digital Factory has produced several new digital products whose conception and design has been directly influenced by DFI. For example, the Ikofi mobile app for farmers was created by developers who completed the CIDM. Likewise, the Digital Factory uses the template which practitioners in that unit learnt from the Mobile Money Operations course to structure their projects, and the steps they use to create products. Interviewees also mentioned how the DFI courses had assist them in improving BK’s settlements and disputes systems, and in developing a new payments and connection system. Managers also talk of not having to micro-manage employees anymore because of their new levels of skill gained through DFI.

The fact that BK chose to sponsor many of its employees in crucial units to take DFI courses shows how much value it places on these courses and their impact. Moreover, the fact that BK has an internal DFI group which meets regularly clearly suggests that DFI has influenced not only the knowledge, but also the work culture of employees at the bank. As BK transforms even further and continues to be a leader in the digital banking space, it is likely that DFI will continue to be a valuable tool in this regard.

 

3. Impact on Countries

The Case Study on Uganda has shown that DFI has played a significant role in the evolution of the DFS landscape in Uganda over the last four years. By training close to 100 practitioners, many of them working in influential positions within the DFS space, DFI has enabled a critical mass of professionals to emerge and begin to collaborate. As Diana Akullu argues, “The key contribution of DFI is awareness. Most people did not know about DFS before 2016. Most people did not know how it worked. This awareness has led to the improvement of services and the quality of services in the DFS space.” Edwin Byaruhanga, who sees himself as being extremely passionate about DFS in Uganda, feels that the DFI courses stimulated him to develop himself and his country in the realm of DFS. He says the following about the role of DFI in Uganda: “It is not too much to say that they have played a major role. Because if one goes back to the time of the first cohort. I know some people who were there in that first cohort. Before we got this knowledge from DFI, we were lagging behind. So if it was not for the exposure that we got there we would probably still be doing the things the way we used to do them.”

Because many of the Ugandan DFI alumni have been in influential positions in the private sector or in developmental organisations, they have been able to initiate and influence projects which are taking DFS in Uganda to the next level. Projects such as the International Institute of Rural Reconstruction’s partnership with the Ugandan Ministry of Finance to promote rural financial protection; the Rkiga SACCO’s shift from a paper-based system to a digital system which has resulted in significant savings for farmers and created a new demand for DFS in rural Uganda; and the lobbying of the Uganda Bankers Association which resulted in agency banking being adopted in the country. Although still small in number, the DFI practitioners working for regulators have also opened doors to their institutions, so that meaningful change can take place. The Ugandan CoP has also played a major role in bringing practitioners from different sectors together (including regulators), especially through the formation of the DFA, ensuring that the DFS regulatory framework keeps up with the fast-changing DFS landscape. The DFA has been able to be influential because of its credibility and the creative and sensitive way it has approached its role in Uganda.

The Case Study on Zambia has shown that DFI has made a number of contributions to the rapid evolution of the DFS landscape in Zambia over the last four years. Firstly, by training a large number of practitioners (153), DFI has had impacts at an individual and institutional level. The fact that many of those working in crucial bodies such as FSD have taken the courses, and then undertaken to ensure that many regulators take the DFI courses, has meant that there is a common understanding of and energy around DFS developing in Zambia, with the potential to make a systemic impact. Through these role-players in particular, DFI has been able to impact DFS regulation and practice, something the newly-formed ADFP should be able to build on in future. Betty Wilkinson, CEO of FSD was unequivocal when asked if DFI has contributed to DFS in Zambia:

“Oh, I would say not even any question because when I arrived here, a lot of people had no idea what this market was about, including the regulators. As a result of them learning about this and having conversations with their seniors, that’s what ended up driving the development of the payments department in the Bank of Zambia. Before that they had three people, now they have nine and it is a proper department with a Director. So that’s actually come on in leaps and bounds. And what’s very interesting for me is that tech is usually a domain that’s largely male in Zambia. What we’re seeing, particularly in the regulator, there’s a significant number of women who are in roles of power and responsibility, both in the regulators and also the person who’s head of the National Pension Authority.”

Joseph Michelo also feels that DFI has made a marked impact: “DFI’s role in Zambia is that it has unified the DFS practitioners in Zambia. It has drawn different individuals from different organisations who might have never met, to share ideas together. The people are coming from fintechs, from credit bureaus, from consumer protection units, Ministry of Finance in charge of digitisation. So it has brought in quite a number of professional players who understand what the digital space looks like. If we look at number of people trained, it has been quite a lot, from public, private sector and regulators.” Mulenga Chileshe feels similarly: “DFI has brought in a whole new dimension in terms of how individual members view the future of DFS in Zambia. There is an entirely new crop of professionals that are committed to be the pioneers for DFS in the country.” The formation of the Association of Digital Finance Practitioners (ADFP), which arose out of the DFI community of practice, has played a major role in bringing DFS practitioners from different spheres together, with the potential to become a major driver of collaboration.

 

[1] DFI have surveyed thousands of students over the last four yours at the start and end of their courses to measure change in knowledge, awareness and confidence. Retention and application have been surveyed 6, 12 and 18 months after as well as in annual focus groups in select countries.

[2] M&E reports available from DFI include: The Final Pilot Report for 2016-2017, the 2018 Focus Groups and Country Reports; the 2019 MMR report including the 5 case studies mentioned above.

[3] Extract of 2019 MMR Report – 3^rd party M&E report researched and authored by Development Works Changemakers

All reports were authored by Development Works Changemakers.