From top left: Aakanksha, Giselle Melo, Saima Fancy, Gayatri Sarkar, Patricia Thaine, Mia Shah-Dand, … [+]
Women in AI Ethics
This post was written in collaboration with Ryan Panela. Disclosure: I am on the Board of Women in AI Ethics and also work for Altitude Accelerator.
On October 24, 2024, Women in AI Ethics launched its inaugural Responsible AI Ecosystem Summit in New York City, hosted in collaboration with the Canadian Consulate General in New York and Altitude Accelerator. A delegation of investors, researchers, founders, practitioners and innovators, including strong participation from Canada, convened to build a responsible and ethical AI ecosystem advocating for human-centered design.
Mia Shah-Dand, CEO of Lighthouse3 and Founder of Women in AI Ethics emphasized,
“This summit reflects our core mission and key role as a catalyst in the global movement towards sustainable and responsible AI. While AI presents many benefits, there is an urgent need for new AI solutions that minimize risks and ensure benefits from AI are distributed equitably. We believe that a thriving responsible AI ecosystem is the pathway to new opportunities, economic growth, prosperity, and vibrant technological futures that include all of us.”
The event tackled the most pressing issues including support for Canadian startup founders, how generative AI is compelling technology companies to address issues when it comes to safety, fair model representation, the fundamental right to data privacy, and mitigating risks to women and minority populations especially as AI advances. While awareness of the disproportionate funding against the women and minority founders has increased, very little has changed in the last 5 years. Economic equity continues to be an uphill battle.
Integration Of Trust In AI
One37 Fadwa Mohanna, Safia Morsly-Fikai, Consultate General Canada, NY
Women in AI Ethics
Safia Morsly-Fikai, Trade Commissioner, Consulate General of Canada in New York spoke with Fadwa Mohanna of One37, which enables businesses and users to connect, exchange, and verify data. Through verifiable credentials technology, Mohanna has developed a secure identity-based authentication, reducing fraud and ensuring seamless, safe and secure AI-powered user interactions. The organization allows for incoming and outgoing information between business and consumers where customer data remains in their digital wallets.
Mohanna spoke about the integration with IBM’s WatsonAI which she believes will transform business operations. Integration of chatbots of bill payments, booking services and dispute resolutions across financial institutions can still be executed while preventing the transmission of personally identifiable information (PII).
AI Safety And Alignment
Aakanksha, Cohere, and Saima Fancy, Ontario Health
Women in AI Ethics
AI safety focuses on ensuring AI systems operate safely and ethically without causing harm or unintended consequences. As large language models become more prevalent, choosing the right training data for safety alignment is crucial. However, this raises the question: Alignment to what? Whose values? Despite global AI use, safety measures often prioritize Western-centric concerns and homogeneous, and contexts to one-language.
Data diversity and multilingual datasets play a vital role in minimizing AI-related harms. The challenge lies in optimizing large language models for various languages and cultural preferences while addressing both global and local issues. Saima Fancy, Senior Privacy Specialist at Ontario Health discussed these concerns with Aakanksha, currently a member of the technical staff in the Safety team at Cohere, when it comes to safeguarding AI systems designed to serve global populations.
Aakanksha confirmed that most of today’s models are trained primarily in English, German and French and it’s increasingly clear there is linguistic inequity and lack of diversity in most training data. Cohere is dedicated to developing models trained on multilingual data and enlists participation from many countries, across cultures and languages to ensure representation.
How does Cohere ensure context to the original language is maintained and not inadvertently misrepresented in translation? If sentences have the same meaning across languages, embedding models will demonstrate high semantic similarity between the pieces of text even though they are in differing languages. Cohere’s models do not contain an automatic translation prior to linguistic encoding which is meant to preserve the specific nuances across languages. This can be useful when it comes to translating or summarizing medical notes and, as Aakanksha noted, communication of medication outcomes and treatment plans in a manner that is culturally appropriate to the patient.
Is there a risk to the data that is collected for training? Cohere does not store data but rather the raw model is provided to the enterprise for their respective use and fine-tuning. In addition, synthetic data is used to train the multilingual models, as it is necessary for data augmentation and may, in the process, remove some of the biases that are present in human data.
The Dichotomy of AI and Privacy
Privacy is a fundamental right. It is essential to an individual’s dignity and enjoyment of personal freedoms. The right to privacy is enshrined the federal laws of most countries around the world. The AI boom and the proliferation of large language models (LLMs) poses new challenges for privacy. The rapid scraping of internet information increases the chances of personal information becoming part of models’ training data, therefore presenting a serious risk of private information being leaked through model output and exposed through third-party hacks. Individual control over personal information now seems elusive. Two leading privacy-centered technology companies, Proton and Private AI, shared how their respective organizations are protecting user privacy and ensuring security of their personal information.
Proton
Jurgita Miseviciute, Proton
Women in AI Ethics
Jurgita Miseviciute is the Head of Public Policy and Government Affairs at Proton. She spoke about the dawn of Proton and Andy Yen, CEO’s vision for their organization. Proton is a Swiss-based technology company that is built on the philosophy that the data privacy is at the forefront of bettering the internet and technology landscape. Proton was born in 2014, an initiative launched by Andy Yen and a group of scientists who met at CERN, the European Organization for Nuclear Research.
After Edward Snowdon’s revelations about the NSA surveillance program, the founders launched a crowdfunding campaign for one simple vision: “to remake the internet in a way that is private by default, and serves the interest of all of society, and not just the interest of a few Silicon Valley tech giants.”
“Proton has since grown to a global privacy powerhouse. What started as Proton Mail, now the world’s largest encrypted email service, has blossomed into an ecosystem of privacy-centric products including Proton VPN, Proton Calendar, Proton Drive, and Proton Pass. These services have become the vanguard of a movement that puts user privacy first, protecting over 100 million accounts worldwide and employing over 500 people.”
Mail was the first product launched by the company. Proton is end-to-end encrypted, and the company does not hold the keys needed to decrypt user data, which means it is unable to access user data even if it wanted to. Miseviciute stresses “This allows the user to take back their privacy. Everything that is written in email stops with the user.” The company structure removes their position as the middleman between the user and potential government access, effectively stopping the latter from requesting access to user data.
Proton believes privacy is integral to AI models, and rather than AI models adding privacy components after the fact, they should be integrated into the technology from the beginning. For all Proton products, AI data is secure and will not be used for training without consent.
Proton’s belief is that AI models should be open source to distribute overall capacity, instead of being in control by a handful of organizations.
Private AI
Patricia Thaine, Private AI
Women in AI Ethics
Patricia Thaine, Co-Founder & CEO of Private AI. Private AI was named a 2023 Technology Pioneer by the World Economic Forum. Thaine was on Maclean’s magazine Power List 2024 for being one of the top 100 Canadians shaping the country.
Thaine pointed to the exposure of PII and its use in AI models as the main concern in modern technology. PrivateAI is an AI model aimed at identifying and suppressing PII through redaction and pseudonymization. While the idea is not new, PrivateAI uses a structure that increases overall performance and accuracy. Thaine indicated that regular expressions, which identify common patterns in PII expression and out of the box models are not effective, as she states, “Humans do not always write or speak in a manner that can be accurately interpreted by their models; however, the integration of AI into these systems can help to improve overall performance.”
To build their model it begins with identifying what is PII and what risk does each type of information contain. Private AI’s model not only allows the user to select the type of PII to detect and remediate; it also allows them to keep up to date with all privacy legislations.
When it comes to model training with sensitive information, Thaine reveals that data can be decoded from the embedding space and reveals much of the sensitive information which was used for training. She adds that PII in training data should be cleaned and/or anonymized prior to model training.
PrivateAI can now be integrated with other LLM models such has ChatGPT. Thaine says that PII is redacted prior to ChatGPT input; output is received from ChatGPT and only then is where PII is reinserted into the output text. LLMs do not gain access to sensitive information.
The State of Investing in Diverse Founders
Hessie Jones, Altitude Accelerator; Gayatri Sarkar, Advaita Capital; Giselle Melo, MATR Ventures
Women in AI Ethics
Artificial Intelligence (AI) has been a significant area of research and investment for many decades. In 2023, venture capital (VC) investment in Generative AI reached $21.3 billion globally compared to just $1 billion in 2018.
Despite the surge in funding, VCs face a new landscape of regulatory uncertainty and growing ethical risks related to the development of LLMs. Founders are faced with risk of market saturation and a challenging path to monetization especially in developing a moat around LLMs which are controlled by a few large organizations.
In addition, women and minorities are drastically underrepresented in venture capital while women-led tech startups only get a fraction of total tech VC funding, which threatens to further exacerbate the inequity in this space.
I led this discussion with Giselle Melo, Managing Director, MATR Ventures and Gayatri Sarkar, Owner of Advaita Capital.
The current landscape where women and minority founders are disproportionately funded in the startup technology space persists. While that was highlighted at least five years ago, not much has changed. Overall, while there has been increased awareness and some initiatives to support underrepresented founders, the statistics show that significant disparities persist, and in some cases, have worsened over the past five years.
According to this Docsend Report, when it comes to pitch decks, slide-by-slide attention from investors varies greatly amongst teams of different demographics. In 2023, VCs spent 66% more time on all-female team sections compared to all-male team slides. VCs also spent the most time on minority teams’ team slides, 20% more than all-white teams. What was not surprising: all-male fundraising asks sections received 25% more time than all-female sections. For many founders, this is a function of an investment sector that has been dominated by males for many years.
As female, persons-of-colour VC fund owners, the statistics are not surprising, however both contend that most deep tech companies do not have a DEI (Diversity Equity and Inclusion) agenda. Melo, a former exited founder with a 13-year track record, leading software systems, machine learning design, and engineering across multiple sectors, acknowledged that MATR is a performance-based fund investing in late seed to Series A deep tech software companies led by inclusive teams.
According to DocSend Report, the gap in dollars raised between all-female and all-male teams widened for the second consecutive year. All-female teams with minority members saw the most significant increase in fundraising times while securing the least amount of capital among all demographics. On average, all-female teams raised 43% less than their all-male counterparts, while diverse teams raised 26% less than all-white teams.
For Sarkar, the owner of a fund that stands alone as a female-led, person-of colour-led fund at the series B stage, she has seen zero deals from female founders. Sarkar recognizes there is still more work that needs to be done. She reflects,
“Many women find themselves having to IPO their companies after Series A and B rounds because they struggle to raise funds for Series C and D. The lack of women writing larger checks is a significant issue. Venture capital, once a nascent asset class, has evolved but it remains challenging to secure a spot on the cap tables of certain firms.”
And while having a champion that vouches for your company can make all the difference, Sarkar stresses that the hurdles they face as a growth fund when it comes to board approvals are symptoms of a broader systemic problem. One issue is the scarcity of women leading growth funds and when she started Advaita, she was advised to raise an early-stage fund.
For Melo, when asked about the challenge of closed networks and who decides who gets to be on the cap table, she acknowledged at Series A she has the flexibility to decide. With the pervasiveness of applied AI and deep tech, her robust network of deep tech subject matter experts and commercialization leaders who may also be investors provides an additional layer of skill sets to interrogate models, scrutinize the technology, and ask the hard questions. This provides a unique service to help derisk investments during the due diligence process and support the growth of her portfolio companies. It also creates opportunities for investors to access untapped investment opportunities.
While Sarkar and Melo represent funds with the intention to bring more diversity into the tech and investment ecosystem, clearly this is an uphill battle that will take time.
Summit Reflections
Women in AI Ethics
Women in AI Ethics
The learning, the conversations, the successes, the awareness and the numbers that showed up for this important event painted a clear picture of the enormous change that will emerge in the coming years. Responsible AI and ethics are now mainstream. But it will require data literacy, education at all levels within industry, across sectors, across all demographics and investment in resources to make startups truly AI ready. Investors, who are used to investing in companies expecting 10X returns in short order need to realize that AI will take time to return a profit. In the meantime, they can influence development of better solutions that minimizing harmful outcomes and as per Mia Shah-Dand’s vision, create a “pathway to new opportunities, economic growth, prosperity, and vibrant technological futures that include all of us.”
About: Ryan Panela is a PhD student in Cognitive and Computational Neuroscience within the Department of Psychology, University of Toronto and Rotman Research Institute, Baycrest Academy for Research and Education and in Data Science and Artificial Intelligence within the Department of Computing, Goldsmiths, University of London. In his research, he emphasizes the critical importance of responsible AI implementation and use across research domains, championing transparent methodologies from experimental design through analysis and ensuring ethical considerations guide all stages of AI deployment in clinical research settings.
