It is no longer a surprising statement that the potential of ChatGPT is ever increasing and now evaluation of ChatGPT has been moved from generalized use cases to more complex and much more specific use cases. And no wonder that this is also resulting in more innovative ways of dealing with complex use cases using ChatGPT and getting more insights in paradigms of LLM’s.
Now a days, it is also well known that the OpenAI’s API has been an impactful move in a way that LLM’s application can be realized and brought to life. Besides API, creating a custom GPT is also one of the useful offerings from OpenAI. While ChatGPT API’s can be used to achieve more complex use cases, custom GPT does not fall short to achieve something that is crisp and well defined.
But what are the challenges to use the responses from the ChatGPT and build custom applications requiring more structured data? How do we deal with “not so simple” structured and unstructured data, with as little code as possible and use ChatGPT to get things done. Perhaps, that eventually lead us to understand the intricacies of ChatGPT and build up a strategy that revolves around your case. While there are many more useful features of ChatGPT such as Assistance and Fine-tuning model, here we explore how to make a strategy that helps to achieve result just using ChatGPT. Here are a few observations made, based on some experiments to achieve certain specific use cases.
Alright, so, let’s consider that you are dealing with PDF, Excel sheets, Images etc. Your PDFs contain somewhat complex data. Which means they are not simple receipts or bunch of text-only data. It is a mix of data such as text that is spread across the page and a table with bunch of rows and columns. (There can be more complex data such as multiple tables and images). And Excel sheets contain multiple rows and columns with say unformatted dates and numbers etc. And now you want to use the information inside such a mix of structured and unstructured data in a way that not just gives you very specific insights but should also be able to give you responses that are structured and consistent. Now, how can we use ChatGPT to achieve this?

1. Using ChatGPT base model (Zero-Shot Prompting):

It is always a good idea to give it a shot and see how well ChatGPT understands the context you want it to recognize by using ChatGPT the way it was meant initially to be used. Use Zero-shot prompting, where you can just feed the data and form a query so that ChatGPT understands the context and returns the response in the format you wanted. Your data may not be used to train the model and the model does not know how such data can be responded with. For example, you can attach the PDFs and Excels and let ChatGPT interpret the contents of the file. But do let GPT clearly know what you expect it to do.
In such a situation, it won’t disappoint you and GPT will give you what you wanted in most runs. But soon you will realize that GPT stops responding to certain information. For example, it will complain that it cannot read PDF, have some trouble with OCR or extract data from Excel. If it succeeds in that, then sometimes it will start hallucinating. Time taken by GPT will be good but will mostly vary based on length of the Prompt statements. One advantage is that OpenAIs API can be used but the response will not be consistent. So, it will be hard to use or integrate with internal API’s.
But this will be a good step to understand behavior of GPT w.r.t your use case. One useful takeaway from this step would be Prompt statements.

2. Custom GPT (one shot or few shot prompting):

As a next step, you could create a custom GPT. You can breakdown the Prompts from zero shot and give better context to Custom GPT. Custom GPT can then be trained with the data in the files. This would result in better accuracy than zero shot prompting. Moreover, there will be a free hand to train the model for your custom data and get the precise output. Using keywords such as “Extract”, “Interpret”, formatting keywords such as JSON, CSV etc. helps model to know precisely what you are expecting.
While this is a progressive step as compared to Zero shot prompting, some issues will still be the same as they were in previous step of Zero shot prompts. Comparatively, accuracy will be improved, and the model will be able to respond with your expectations. Response time from the GPT will be very impressive. You will get results in no time as this model was sharply trained for what you expect.
But OpenAI, at least till date, does not provide API for custom GPT. So, even though this is a very useful feature of ChatGPT, you still don’t have a comfortable way to integrate or automate your tasks.
Nevertheless, what you will gain with this would be more precise multiple prompts that will now act as steps for model.

3. Using ChatGPT base model – Multiple prompt + Code:

So far you would have multiple prompts that can act like precise instructions to GPT. What if you use API’s and use multiple prompts? Well, that would perhaps yield a better result from GPT compared to the above 2 steps. One observation has been that you might have to use external libraries to read the data from PDF, Excel, Images etc. As the result from such tools are more consistent than the result from GPT. The extracted data can then be fed as prompt text and instructed GPT on what to do with it. You can also request GPT to return fixed format JSON, CSV, bunch of specific strings etc. For e.g: you can request a response for the prompt in JSON format using parameter response_format={ “type”: “json_object” }, in API. [Reference].

A more interesting feature of ChatGPT is function calling. You can provide a JSON schema to GPT that states the name of the function (Hypothetical or Real.) and expected JSON format that would be the input to the function. ChatGPT will then provide you with such JSON as response to your prompt. [Reference]. What makes this feature interesting is the ability of ChatGPT to respond in JSON format that can also include arrays and many objects. This makes GPT easy to use and integrated with custom API’s. One example schema that include arrays and object in JSON is below:

Function_name = [


“type”: “function”,

“function”: {

“name”: “functionName”,

“description”: “Describe what this function will do.”,

“parameters”: {

“type”: “object”,

“properties”: {

“Item”: {

“type”: “array”,

“I1”: {

“type”: “object”,

“properties”: {

“Key 1”: {

“type”: “string”,

“description”: “Describe Key 1”


“Key 2”: {

“type”: “integer”,

“description”: “Describe Key 2”



“required”: [“Key 1”, “Key 2”]




“required”: [” Item “]



} ]

Such JSON schema can then be used to streamline result from GPT and GPT will start responding with fixed format of the result. Also, the amount of code can be significantly reduced with multiple shot prompting and asking GPT to share the result in fixed format data. Such data can also be used in databases and reused later for various purposes, and thus overcome ChatGPT’s drawback of remembering the history.
Though, this step will be huge improvement from previous steps with fixed format result, consistency and accuracy still can be a concern. Also, response time would not be impressive especially when you are using API’s. Expect some delays and slow processing of data when your prompt sequence progresses from first to last in case of multiple prompts with data concatenation. Moreover, there will be some charges that must be paid, so prompts should be used consciously.
But still, this can be a good step, if not an entire solution, that can help you minimize efforts to achieve the fulfilment of the use case. If there are better prompts derived from the first 2 steps, then tokens can be minimized.

Although ChatGPT deals with structured and unstructured data very well, we still need some strategy to use the power of GPT in a way that can handle specific use cases and minimize significant efforts.

And one of the strategies could be visualized as below:

The observations can be briefly summarized:

ChatGPT base model (Zero-Shot Prompting) Issues with consistency and unformatted/unnecessary statements in response, hallucinations. Predict GPT behavior for specific use case.
Improvement in prompting. Can be used with API. Good response time in general.
Custom GPT (one shot or few shot prompting) Hallucination requires specific instructions to handle corner cases, inconsistency if provided with more instruction and information. No API so far. Improved consistency with response, can train the model, request for fix format. Mostly, impressive response time.
Using ChatGPT base model – Multiple prompt + Code Some coding required, Precise instructions, Payment and Rate limits. Slow response especially if there are multiple prompts with concatenated data input. Fixed format result, improved consistency, no need to train the models, API integration possible

Note: Observation are made using GPT model gpt-3.5-turbo and its sub versions for a specific use case and summarized.

About the author:

Mohanish is a Lead Software Engineer with around 16 years of experience in various domains such as Linux based applications, Security and Android. He is also working in ML/DL and AI to analyze various use cases and provide efficient solutions.
Siri AB is Gothenburg, Sweden based organization with expertise in Automotive, Telecom, IoT, ML/DL and AI engineering. Siri AB can provide you with consultation support for implementing various use cases with LLM’s such as ChatGPT.

Leave A Comment