Hjem Blogginnlegg When and why using (Go) function calls in GPT
Blogginnlegg
Blogg

When and why using (Go) function calls in GPT

The new generative AIs are being used for all kinds of purposes. It is a time for experimentation. And caution.

At Computas, one of the things we’re using OpenAI’s GPT models for is information transformation. For some clients we transform semi structured information with a lot of prose into structured information. Similar to what we did with Natural Language Understanding (NLU), entity extraction (NER) and slot-filling prior to the arrival of these new powerful LLMs. Frequently, the need is to use that structured information to do some background processing. It might be something that changes state in some backend system. Or do a search. Or some enrichment. The result from this processing we in some projects need to put back into prose for human consumption.

Let’s say for example you have voice or text messages representing purchasing orders. A message may say “Hi, we don’t need the screws after all. We’ll do it with clout nails instead. So perhaps 250 30 mm. And the same with 50 mm. Both 2.8. Send it with the same package if you please. Dick”. The challenges here are many; what has actually been ordered here? What types of items. How many. What size. And which existing order is being modified with what exactly? GPT gives us an entirely new solution space for this, which shows great promise.

Function calls

One task for GPT could be to ask it to fill out a structured form representing an order. The form could be a schema where salient fields are “item_type”, “amount” and “size”. With that information in hand a background process could map these fields to a formally defined item registry where a unique identifier pins down the item in question. This could then be used by an order system further downstream. OpenAI’s GPT API gives us some interesting choices when we want to find the information nuggets in a message. We can use vanilla prompt engineering and ask GPT to reply in a structured way;

Read this order; "Hi, we don't need the screws after all. We'll do it with clout nails instead. So perhaps 250 30 mm. And the same with 50 mm. Both 2.8. Send it with the same package if you please. Dick"

Please find each item, the amount, size and unit and put each  on a line on the form 

item type : size : unit : amount

where GPT (ChatGPT in this example) answers

Item type: Clout nails
Size: 30 mm
Unit: 2.8 (assumed to be the diameter or gauge of the nails)
Amount: 250

Item type: Clout nails
Size: 50 mm
Unit: 2.8 (assumed to be the diameter or gauge of the nails)
Amount: 250

As you can see, this free form prose comes with challenges if you’re to use this as input to a backend function. Neither did GPT produce single colon separated lines per item as we requested, nor is the Unit field suitable.

Recently, OpenAI has provided us with a function call interface as an alternative. This is a sequence of API calls where you with the initial prompt provide a function signature. Illustrating in Go, and using Alexander Baranov’s dependable go-openai library we define functions;

resp, err := client.CreateChatCompletion(
  context.Background(),
  openai.ChatCompletionRequest{
   Model:     openai.GPT3Dot5Turbo0613,
   Messages:  messages,
   Functions: []openai.FunctionDefinition{*functions},
  },
 )

GPT decides itself whether some of the text fits the function signature(s) and might respond with a filled out parameter section for the call. We, the developer, decide how to act upon this, possibly making a backend call with the parameters provided, here illustrated with the callFunction()

if resp.Choices[0].FinishReason == "function_call" {

  messages = append(messages, resp.Choices[0].Message)

  functionToCall := resp.Choices[0].Message.FunctionCall.Name
  args := resp.Choices[0].Message.FunctionCall.Arguments
  functionResponse := callFunction(functionToCall, args)

  message3 := openai.ChatCompletionMessage{
   Role:    openai.ChatMessageRoleFunction,
   Content: functionResponse,
   Name:    functionToCall,
  }

  messages = append(messages, message3)

 }

Then you provide the result of the backend call to GPT.

resp, err = client.CreateChatCompletion(
  context.Background(),
  openai.ChatCompletionRequest{
   Model:    openai.GPT3Dot5Turbo0613,
   Messages: messages,
  },
 )

GPT then responds in this third round with a natural language response where the background call’s response is baked in as prose. A python version is in the OpenAI doc. The technicalities of the procedure in python has been well covered by various blogs and articles.

Why and when?

So what are the pros and cons of using this new way of calling backend services? What we find is that the pros are

  • Function calls are a succinct way of filling out function parameters for a backend call in a dialogue. There might be less prompt engineering and clutter.
  • You might have a better chance of filling out required parameters and valid json.
  • You get a natural language response back to the end-user out of the box.

There are some cons, or at least things to be cautious about however

  • GPT isn’t given a lot of information about the call. As the snippet above shows, you get to describe the function and the parameters, but it leaves ample room for misinterpretation. Given a temperature above zero, the responses are stochastic and you might find that a particular GPT response doesn’t capture the meaning of the function response sufficiently.
  • Similarly, you might find that GPT is better at populating the parameters you need by providing context and guidance in the prompts rather than as part of the function call specification
  • Since the function call is prepared for you by GPT, you might gradually slide towards less human validation and less checks before you call backend functions that may have real world consequences. You need to be careful.
  • You might want to call more than one function as a response to a prompt. The API gives you limited freedom. You might want more flexible semantics.
  • In some cases you might not need to provide a natural language response after the function call. For the message example provided above and with the client we work with there, that need is not prominent.
  • You need to use specialized models that are quickly deprecated.

The API and technology is in great flux of course. Next month there might be new, even better ways of dealing with these classic NLP challenges.

Happy coding 🙂

When and why using (Go) function calls in GPT was originally published in Compendium on Medium, where people are continuing the conversation by highlighting and responding to this story.