To tease out PromptBro’s capacity to help users create unique LLM experiences, I decided on a new side-project: creating a minimal legal chatbot.
By now (August 2024), the risks of using LLMs for legal research are well known. The tale of the American lawyer who submitted hallucinated cases to Court has become part of legal folklore.
On the other end of the spectrum, there are already a number of dedicated startups in the crowded LegalTech eco-system surrounding the use of LLMs.
The highest profile (due to its huge funding and clientele including Allen and Overy and PwC) is likely Harvey AI, with a strong focus on confidentiality and safety.
In light of this background, what might a suitable side project be?
It is too ambitious to do something similar to Havery AI. Hallucination is a major issue, and trying to solve this is beyond the scope of a side project.
Instead, a more promising angle will be to create a LLM to help lawyers “bounce” ideas off with it.
For example, say I need to advise on an issue in a new area, where I am not sure what the relevant search terms may be. Can I fine-tune a LLM to help me find as many similar search terms, which may (for example) help me find cases in other common law jurisdictions?
This is something one can already do with “generic” LLM, such as ChatGPT (I found the phrase “no property in a witness” that way). But perhaps I can enhance the experience with appropriate fine tuning.