ChatGPT in Software Development: Our Client's Top Questions

1. Does Valudio use ChatGPT?

Yes, Valudio has integrated ChatGPT into the day-to-day responsibilities. This technology has become a helpful tool in our routine tasks, pivotal in automating and simplifying various aspects of our work.

2. What projects are you working on for your clients that involve ChatGPT? 

We just finished working on a proof of concept for a Swiss manufacturing company looking to create a digital assistant for their customers. This assistant is an in-house chatbot to answer customer questions about their products.

To build this chatbot, we've been importing product marketing materials from various formats like PDF, HTML, and Excel and integrating them into the chatbot's knowledge base. We also incorporated media and voice files utilizing AI services for audio transcription.

The chatbot will serve two primary purposes:

1. Website Integration: It will be integrated into our client's website, providing a quick and convenient way for customers to get product-related information when they visit the site.
2. Metaverse Exploration: Additionally, our client is exploring the development of a metaverse, a digital environment resembling a salesroom where customers can interact with a digital assistant.

Currently, we focus on building an essential interaction: customers ask questions, and the chatbot provides answers. We have yet to implement complex features like multistep conversational workflows.

3. Do Valudio’s developers use AI to review code or anything similar? 

Yes, our developers make use of a platform known as GitHub Copilot. GitHub Copilot is a tool and subscription service provided by GitHub, which can be integrated into your software development environment. While actively coding, it suggests code segments based on what you've already written. This can be particularly handy for tasks like creating test cases or generating test data, among other uses.

4. Is the artificial intelligence system used by Valudio separate from OpenAI's ChatGPT?

ChatGPT is a product that utilizes generative AI technology, specifically the GPT (Generative Pre-trained Transformer) architecture developed by OpenAI.

GitHub Copilot is powered by a model called Codex, which is also based on the GPT architecture. While Codex and ChatGPT (like GPT-4) share the same foundational technology, they are trained for different purposes. Codex is specialized for code generation and assistance, while ChatGPT is designed for more general conversational tasks.

Answering the question, GitHub Copilot, which is used by Valudio, is built using ChatGPT. They share the same foundational technology. So the artificial intelligence behind it is not that different at all.

Currently, the most well-known models include ChatGPT-3.5 and 4, but there are less renowned models tailored for specific use cases like code completion, reasoning, mathematics, etc. Some of these competing models are even open-source. One limitation of ChatGPT and Azure OpenAI is the requirement to run them on their cloud platform, which may not align with everyone's preferences.

An open-source alternative is LLama 2 from Meta. This model can run on-premises without depending on OpenAI or Microsoft Azure.

5. How is ChatGPT used in software development in general? 

ChatGPT and similar language models can be valuable tools in various aspects of software development. Here are some ways in which ChatGPT will help when software is being developed:

Code Generation

ChatGPT can assist developers in generating code snippets for specific tasks or functions. Developers can describe their requirements in plain language, and ChatGPT can develop code templates or provide code suggestions to streamline the development process.

Debugging and Troubleshooting

Developers can describe code issues or error messages to ChatGPT, which can help identify potential causes or suggest debugging techniques. It can serve as an interactive debugging assistant.

Documentation Assistance

ChatGPT can assist in generating documentation for code, libraries, or APIs. Developers can provide descriptions, and ChatGPT can help create clear and concise documentation.

Code Review and Quality Assurance

ChatGPT can assist developers in code review by proposing improvements, identifying potential security vulnerabilities, or providing best practices for coding standards.

Automated Testing

ChatGPT can help create and manage test cases for software testing. It can generate test scenarios based on user input and expected outcomes.

Project Management

ChatGPT can assist in project management by helping to generate project plans. It can also assist in developing reports and documentation related to project management.

Natural Language Processing (NLP)

ChatGPT is a versatile tool in the NLP domain. It can power applications like chatbots and virtual assistants. While it can also assist with tasks like language translation, it's worth noting that there are more specialized models that will offer more accuracy for language-specific tasks.

6. Can ChatGPT maintain context? 

It’s worth breaking this question down into different sub-questions, as it’s important to understand what the ChatGPT context is as well as the all-important whys and hows.

What is the ChatGPT context?

In the realm of conversational AI, context is king. But what exactly does “context” mean in ChatGPT? Context can be thought of as the model's short-term memory, capturing the immediate history of a conversation. In contrast, the model's long-term memory represents the vast knowledge it acquired during training.

ChatGPT: Context vs. Model Knowledge

ChatGPT's Context

• Definition: Context refers to the immediate chunk of conversation that ChatGPT “remembers” during an interaction. It's the recent history of the conversation.
• Duration: It's temporary. Once the session ends or exceeds a certain length, ChatGPT loses this context.
• Purpose: Context allows ChatGPT to generate coherent and relevant responses based on the ongoing conversation. For instance, if you ask about the weather in Paris and then say, “How about tomorrow?”, ChatGPT uses its context to understand you're still referring to Paris's weather.

ChatGPT's Model Knowledge

• Definition: Model knowledge encompasses all the information, patterns, and data that ChatGPT was trained on. It's the vast amount of text and information it has seen during its training phase.
• Duration: It's permanent for a given version of the model. This knowledge doesn't change or update in real time. For example, if ChatGPT was last trained in 2022, it wouldn't know about events from 2023 unless a new version is trained with updated data.
• Purpose: Model knowledge allows ChatGPT to answer a wide range of questions, generate creative content, and engage in diverse topics. It's the foundation that enables ChatGPT to understand and generate human-like text.

While context is about the immediate conversation and helps ChatGPT maintain relevance in a chat session, model knowledge is the vast reservoir of information it pulls from to answer questions and engage in discussions. Think of context as the short-term memory of a conversation and model knowledge as the long-term memory built from its training data.

Why do you need the ChatGPT context?

Beyond recalling conversation history, the context in ChatGPT serves another vital purpose: it can introduce information that isn't part of the model's long-term knowledge. You might have encountered instances where ChatGPT wasn't familiar with certain topics. For software development companies like Valudio, this presents an opportunity. When integrating ChatGPT into custom software or building a bespoke chatbot, this technique can be employed to feed the bot information on topics it hasn't been trained on. This technique is called “priming”.

For instance, if you're using ChatGPT and you want it to answer questions about a fictional company or a topic it wasn't specifically trained on, you might start the conversation by providing a brief summary or details about that topic. This “primes” the model to use that information in its responses.

However, it's worth noting that while priming can be effective, it has its limits. The model will still rely heavily on its training data and inherent behaviors, and the priming context can only influence its outputs to a certain extent.

How can you extend the knowledge of ChatGPT?

You cannot pass a book to ChatGPT to memorize. As discussed above, sometimes you have to train (extend the knowledge) of ChatGPT to fit your specific needs. Below, we outline various techniques to enhance ChatGPT's capabilities. These methods are grouped based on the stage of model development at which you'd introduce the new data.

Internal Model Adjustments

Techniques that involve modifying the model's internal parameters or architecture. Therefore, you have to change the model itself.

• Fine-tuning: Further training on a task-specific dataset.
• Regularization Techniques: Applying methods like dropout or layer normalization during fine-tuning.
• Custom Architectural Modifications: Changing the model's structure to better suit specific tasks.

Prompt-based Guidance

Techniques that rely on the input prompt to guide the model's behavior without changing its internal parameters. 

• Priming: Providing context or information in the prompt to guide the model's responses.
• Few-shot Learning: Giving the model a few examples of a task to guide its behavior.
• Task-specific Prompt Design: Crafting prompts to be more explicit or directive.

Aggregation and Redundancy

Techniques that involve using the model multiple times or with other models.

• Model Ensembling: Combining outputs from multiple models or multiple runs.

External Knowledge Integration

Techniques that integrate external sources of information with the model's outputs.

• Knowledge Integration: Merging external databases or knowledge bases with the model's responses.

7. How safe is ChatGPT? 

ChatGPT serves as a user-facing application used in daily interactions. Additionally, an API lets you incorporate ChatGPT's capabilities into your applications. Given Microsoft's ownership stake in OpenAI, they offer access to ChatGPT through their Azure cloud platform. The distinction lies in data security: with ChatGPT, any input lacks protection, and your data isn't safeguarded – it can be used without restrictions. Conversely, Microsoft Azure ensures that your input data remains within your Azure subscription, guaranteeing it won't be shared with others.

You can also restrict ChatGPT's access to external knowledge. In other words, you can instruct it to rely solely on the data you provide. This reduces the likelihood of receiving incorrect answers. If you inquire about something outside ChatGPT's knowledge or information you have yet to supply, it will respond by saying it lacks ability. This is precisely what we're implementing in the chatbot we're developing for the Swiss company.

8. What are the use cases for ChatGPT that Valudio promotes? 

In today's world, there's a widespread effort to incorporate generative AI into applications, and the potential is immense. The ability to automate various tasks is a game-changer. In the past, we developed chatbots that often involved dealing with static, complex model training. However, with ChatGPT, the complexity has been significantly reduced, making it a straightforward and accessible tool for everyone to build upon. If you’re thinking of ways to help drive your business forward with ChatGPT, take a look at how to use ChatGPT for Software Development for more information. 

9. Can ChatGPT replace human developers? 

This is a question that is on everybody's lips. AI is already increasing productivity, which will only improve in the future, but will it replace human developers entirely?

It brings to mind the ongoing debate surrounding low-code and no-code development. While these approaches enable you to create applications without extensive coding knowledge, they come with limitations. Your ability to build depends on the constraints of the chosen no-code or low-code platform, which may restrict you from implementing certain functionalities. This limitation can be a significant drawback.

Furthermore, when you use such platforms, you often have to share some of your revenue with them. They can be valuable for quickly prototyping or testing small projects, but our advice is to consider building from scratch for more complex and scalable software products.

Actual software development still requires a human touch because it involves communication. The technical aspects are typically manageable, but effective communication can be challenging. Misunderstandings can arise, trust is crucial, and human collaboration plays a vital role in overcoming these challenges. However, the future is full of surprises. Just last year, we marvelled at the capabilities of ChatGPT. Three years ago, COVID-19 surprised the world. Given these unpredictable turns, perhaps it's time to adopt a 'never say never' mindset. Only the future will tell.