Mention generative AI to a group of Chief Procurement Officers and you’re likely to get a mix of curiosity and fear. Curiosity about how AI tools can help make the procurement process more effective, coupled with fear of errors, omissions, and creating a system where the value of human experience and intuition no longer has a place.
I’ve been curious about the potential of generative AI in procurement myself, so I decided to road-test some of the common generative AI platforms and see how they’d help a procurement team with a typical task — creating questions for a Request for Proposal (RFP).
A recent project of mine involved helping a client source a content management system (CMS), so with the RFP I created as a control, I sought to discover what kinds of recommended questions would be generated by different generative AI platforms; my objective was to assess their feasibility in assisting procurement professionals and to identify the strengths and limitations of each.
Three prominent generative AI platforms — ChatGPT 3.5, Microsoft Bing AI, and Google Bard — were given key facts about the business’ needs and asked to generate RFP questions. Why these three? Because they are free and available to anyone. A business considering investing in a for-pay platform is likely to want to test the idea first, and what better approach than trying one of these pioneering platforms? Here are my observations:
ChatGPT
ChatGPT 3.5, developed by OpenAI, exhibited an impressive capability to generate detailed and relevant RFP questions. Its recommendations were clear and aligned with the given prompt, displaying an understanding of the technical requirements and intricacies of how the platform would be used. The suggestions fell under categories including technical capabilities, user experience and interface, integration and APIs, security and compliance, pricing, and licensing. There were multiple questions posed under each category.
This level of detail showcased its ability to comprehend nuanced requirements and translate them into meaningful questions that potential suppliers would need to address.
One of the proposed RFP questions from ChatGPT is my personal favorite to ask of potential suppliers: “Highlight the unique strengths or differentiators that set your CMS platform apart from competitors.” ChatGPT is the only generative AI product suggesting that question be included.
Microsoft Bing AI and Google Bard
Both Microsoft Bing AI and Google Bard recommended questions that were far inferior and less complex than ChatGPT. Bing’s recommendations were worded more like you would expect in a business document (“What are the costs and pricing models of your CMS platform”) while the questions from Google Bard were more general and high-level (“What is your budget”).
Although they both generated some usable questions, neither went far enough in depth or specificity. Procurement professionals using either one of these would need to invest significant time in editing and refining the questions to meet typical business standards.
Key takeaways
The experiment yielded three key takeaways that can guide procurement professionals in effectively using generative AI platforms for RFP question creation:
- ChatGPT 3.5 performed best overall: Of the three platforms tested, ChatGPT 3.5 stood out as the most capable and reliable tool for generating RFP questions. It was able to comprehend complex requirements and produce detailed questions. It produced 24 RFP questions while Bing and Bard output about half that number.
- AI is a great starting point but human intervention is still required: Generative AI platforms offer an efficient way to initiate the RFP question creation process. They can help streamline the initial brainstorming phase and ensure that critical aspects are covered. However, thorough evaluation and customization are necessary to create a well-rounded RFP, so — at least at this point — human experience and input are required to cover all bases.
- Continued tweaking will enhance results in the long run. It’s well-known that machine learning helps AI get smarter over time. So continued use and fine-tuning of the platform you choose will lead to improved performance.
It’s not perfect yet, but the integration of generative AI platforms holds significant promise for RFP generation. This experiment highlighted the prowess of ChatGPT 3.5 in generating detailed and pertinent RFP questions.
While the technology is not without its limitations, it holds the potential to be a great starting point for procurement professionals, especially when they need to create an RFP for a category in which they are not totally familiar. Procurement chiefs should be experimenting with these tools as long as critical evaluation and human touch remain part of the process.
