Skip to content

Commit 231a5ea

Browse files
committed
Add support for Upload API
1 parent 064c11b commit 231a5ea

File tree

17 files changed

+442
-12
lines changed

17 files changed

+442
-12
lines changed

README.md

+11-9
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,8 @@ A Java library to use the OpenAI Api in the simplest possible way.
2020
- [Chat Completion with Streaming Example](#chat-completion-with-streaming-example)
2121
- [Chat Completion with Functions Example](#chat-completion-with-functions-example)
2222
- [Chat Completion with Vision Example](#chat-completion-with-vision-example)
23-
- [Chat Completion Conversation Example](#chat-completion-conversation-example) **New**
24-
- [Assistant v2 Conversation Example](#assistant-v2-conversation-example) **New**
23+
- [Chat Completion Conversation Example](#chat-completion-conversation-example) 🍀
24+
- [Assistant v2 Conversation Example](#assistant-v2-conversation-example) 🍀
2525
- [Support for Additional OpenAI Providers](#-support-for-additional-openai-providers)
2626
- [Azure OpenAI](#azure-openai)
2727
- [Anyscale](#anyscale)
@@ -38,7 +38,7 @@ Simple-OpenAI uses the [CleverClient](https://github.com/sashirestela/cleverclie
3838

3939

4040
## ✅ Supported Services
41-
Simple-OpenAI seeks to stay up to date with the most recent changes in OpenAI. Currently, it supports all existing features until [Jun 6th, 2024](https://platform.openai.com/docs/changelog/jun-6th-2024) and will continue to update with future changes.
41+
Simple-OpenAI seeks to stay up to date with the most recent changes in OpenAI. Currently, it supports all existing features until [Jul 18th, 2024](https://platform.openai.com/docs/changelog/jul-18th-2024) and will continue to update with future changes.
4242

4343
Full support for all of the OpenAI services:
4444

@@ -52,6 +52,7 @@ Full support for all of the OpenAI services:
5252
* Image (Generate, Edit, Variation)
5353
* Models (List)
5454
* Moderation (Check Harmful Text)
55+
* Upload (Upload Large Files in Parts) <span style="color:red">**NEW**</span>
5556
* Assistants Beta v2 (Assistants, Threads, Messages, Runs, Steps, Vector Stores, Streaming, Function Calling, Vision)
5657

5758
![OpenAI Services](media/openai_services.png)
@@ -172,7 +173,7 @@ imageResponse.stream().forEach(img -> System.out.println("\n" + img.getUrl()));
172173
Example to call the Chat Completion service to ask a question and wait for a full answer. We are printing out it in the console:
173174
```java
174175
var chatRequest = ChatRequest.builder()
175-
.model("gpt-3.5-turbo-1106")
176+
.model("gpt-4o-mini")
176177
.message(SystemMessage.of("You are an expert in AI."))
177178
.message(UserMessage.of("Write a technical article about ChatGPT, no more than 100 words."))
178179
.temperature(0.0)
@@ -186,7 +187,7 @@ System.out.println(chatResponse.firstContent());
186187
Example to call the Chat Completion service to ask a question and wait for an answer in partial message deltas. We are printing out it in the console as soon as each delta is arriving:
187188
```java
188189
var chatRequest = ChatRequest.builder()
189-
.model("gpt-3.5-turbo-1106")
190+
.model("gpt-4o-mini")
190191
.message(SystemMessage.of("You are an expert in AI."))
191192
.message(UserMessage.of("Write a technical article about ChatGPT, no more than 100 words."))
192193
.temperature(0.0)
@@ -225,7 +226,7 @@ public void demoCallChatWithFunctions() {
225226
var messages = new ArrayList<ChatMessage>();
226227
messages.add(UserMessage.of("What is the product of 123 and 456?"));
227228
chatRequest = ChatRequest.builder()
228-
.model(modelIdToUse)
229+
.model("gpt-4o-mini")
229230
.messages(messages)
230231
.tools(functionExecutor.getToolFunctions())
231232
.build();
@@ -237,7 +238,7 @@ public void demoCallChatWithFunctions() {
237238
messages.add(chatMessage);
238239
messages.add(ToolMessage.of(result.toString(), chatToolCall.getId()));
239240
chatRequest = ChatRequest.builder()
240-
.model(modelIdToUse)
241+
.model("gpt-4o-mini")
241242
.messages(messages)
242243
.tools(functionExecutor.getToolFunctions())
243244
.build();
@@ -292,7 +293,7 @@ public static class RunAlarm implements Functional {
292293
Example to call the Chat Completion service to allow the model to take in external images and answer questions about them:
293294
```java
294295
var chatRequest = ChatRequest.builder()
295-
.model("gpt-4-vision-preview")
296+
.model("gpt-4o-mini")
296297
.messages(List.of(
297298
UserMessage.of(List.of(
298299
ContentPartText.of(
@@ -311,7 +312,7 @@ System.out.println();
311312
Example to call the Chat Completion service to allow the model to take in local images and answer questions about them:
312313
```java
313314
var chatRequest = ChatRequest.builder()
314-
.model("gpt-4-vision-preview")
315+
.model("gpt-4o-mini")
315316
.messages(List.of(
316317
UserMessage.of(List.of(
317318
ContentPartText.of(
@@ -905,6 +906,7 @@ Examples for each OpenAI service have been created in the folder [demo](https://
905906
* Image
906907
* Model
907908
* Moderation
909+
* Upload
908910
* Conversation
909911
* AssistantV2
910912
* ThreadV2

media/openai_services.png

1.43 KB
Loading

src/demo/java/io/github/sashirestela/openai/demo/ChatDemo.java

+3-3
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ public class ChatDemo extends AbstractDemo {
2828
private String modelIdToUse;
2929

3030
public ChatDemo() {
31-
modelIdToUse = "gpt-3.5-turbo-1106";
31+
modelIdToUse = "gpt-4o-mini";
3232
chatRequest = ChatRequest.builder()
3333
.model(modelIdToUse)
3434
.message(SystemMessage.of("You are an expert in AI."))
@@ -96,7 +96,7 @@ public void demoCallChatWithFunctions() {
9696

9797
public void demoCallChatWithVisionExternalImage() {
9898
var chatRequest = ChatRequest.builder()
99-
.model("gpt-4-vision-preview")
99+
.model(modelIdToUse)
100100
.messages(List.of(
101101
UserMessage.of(List.of(
102102
ContentPartText.of(
@@ -113,7 +113,7 @@ public void demoCallChatWithVisionExternalImage() {
113113

114114
public void demoCallChatWithVisionLocalImage() {
115115
var chatRequest = ChatRequest.builder()
116-
.model("gpt-4-vision-preview")
116+
.model(modelIdToUse)
117117
.messages(List.of(
118118
UserMessage.of(List.of(
119119
ContentPartText.of(
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
package io.github.sashirestela.openai.demo;
2+
3+
import io.github.sashirestela.openai.domain.file.FileRequest.PurposeType;
4+
import io.github.sashirestela.openai.domain.upload.UploadCompleteRequest;
5+
import io.github.sashirestela.openai.domain.upload.UploadPartRequest;
6+
import io.github.sashirestela.openai.domain.upload.UploadRequest;
7+
8+
import java.io.File;
9+
import java.io.FileInputStream;
10+
import java.io.FileOutputStream;
11+
import java.io.IOException;
12+
import java.nio.file.Files;
13+
import java.nio.file.Path;
14+
import java.util.ArrayList;
15+
import java.util.List;
16+
17+
public class UploadDemo extends AbstractDemo {
18+
19+
private String fileName;
20+
private String splitPath;
21+
private String mimeType;
22+
private long totalBytes;
23+
private int totalSplits;
24+
private String uploadId;
25+
private List<String> uploadPartIds;
26+
27+
public void splitSourceFile() throws IOException {
28+
String fullFileName;
29+
String chunkSizeMB;
30+
31+
while ((fullFileName = System.console().readLine("Enter the full file name to upload: ")).isBlank());
32+
while ((chunkSizeMB = System.console().readLine("Enter the chunk size in MB: ")).isBlank());
33+
while ((splitPath = System.console().readLine("Enter the path to put splittings: ")).isBlank());
34+
File sourceFile = new File(fullFileName);
35+
fileName = sourceFile.getName();
36+
mimeType = Files.probeContentType(Path.of(fullFileName));
37+
FileInputStream fis = new FileInputStream(sourceFile);
38+
int chunkSize = Integer.parseInt(chunkSizeMB) * 1024 * 1024;
39+
byte[] buffer = new byte[chunkSize];
40+
int bytesRead = 0;
41+
int chunkIndex = 1;
42+
while ((bytesRead = fis.read(buffer)) > 0) {
43+
File chunkFile = new File(splitPath, fileName + ".part" + chunkIndex);
44+
try (FileOutputStream fos = new FileOutputStream(chunkFile)) {
45+
fos.write(buffer, 0, bytesRead);
46+
}
47+
totalBytes += bytesRead;
48+
chunkIndex++;
49+
}
50+
fis.close();
51+
totalSplits = chunkIndex - 1;
52+
}
53+
54+
public void demoCreateUpload() {
55+
var uploadRequest = UploadRequest.builder()
56+
.filename(fileName)
57+
.purpose(PurposeType.ASSISTANTS)
58+
.bytes(totalBytes)
59+
.mimeType(mimeType)
60+
.build();
61+
var uploadResponse = openAI.uploads().create(uploadRequest).join();
62+
uploadId = uploadResponse.getId();
63+
System.out.println(uploadResponse);
64+
}
65+
66+
public void demoAddPartsUpload() {
67+
uploadPartIds = new ArrayList<>();
68+
for (int i = 1; i <= totalSplits; i++) {
69+
Path partPath = Path.of(splitPath, fileName + ".part" + i);
70+
var uploadPartResponse = openAI.uploads()
71+
.addPart(uploadId,
72+
UploadPartRequest.builder().data(partPath).build())
73+
.join();
74+
uploadPartIds.add(uploadPartResponse.getId());
75+
System.out.println(uploadPartResponse);
76+
}
77+
}
78+
79+
public void demoCompleteUpload() {
80+
var uploadCompleteResponse = openAI.uploads()
81+
.complete(uploadId,
82+
UploadCompleteRequest.builder().partIds(uploadPartIds).build())
83+
.join();
84+
System.out.println(uploadCompleteResponse);
85+
86+
var fileId = uploadCompleteResponse.getFile().getId();
87+
FileDemo fileServiceDemo = new FileDemo();
88+
fileServiceDemo.waitUntilFileIsProcessed(fileId);
89+
System.out.println("The file was processed.");
90+
fileServiceDemo.deleteFile(fileId);
91+
System.out.println("The file was deleted.");
92+
}
93+
94+
public void demoCancelUpload() {
95+
var uploadRequest = UploadRequest.builder()
96+
.filename(fileName + ".tmp")
97+
.purpose(PurposeType.ASSISTANTS)
98+
.bytes(totalBytes)
99+
.mimeType(mimeType)
100+
.build();
101+
var uploadResponse = openAI.uploads().create(uploadRequest).join();
102+
var uploadIdToCancel = uploadResponse.getId();
103+
104+
var uploadCancelResponse = openAI.uploads().cancel(uploadIdToCancel).join();
105+
System.out.println(uploadCancelResponse);
106+
}
107+
108+
public static void main(String[] args) throws IOException {
109+
var demo = new UploadDemo();
110+
111+
demo.splitSourceFile();
112+
113+
demo.addTitleAction("Call Upload Create", demo::demoCreateUpload);
114+
demo.addTitleAction("Call Upload Add Parts", demo::demoAddPartsUpload);
115+
demo.addTitleAction("Call Upload Complete", demo::demoCompleteUpload);
116+
demo.addTitleAction("Call Upload Cancel", demo::demoCancelUpload);
117+
118+
demo.run();
119+
}
120+
121+
}

src/main/java/io/github/sashirestela/openai/BaseSimpleOpenAI.java

+8
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ public abstract class BaseSimpleOpenAI {
3333
protected OpenAI.Images imageService;
3434
protected OpenAI.Models modelService;
3535
protected OpenAI.Moderations moderationService;
36+
protected OpenAI.Uploads uploadService;
3637
protected OpenAIBeta2.Assistants assistantService;
3738
protected OpenAIBeta2.Threads threadService;
3839
protected OpenAIBeta2.ThreadMessages threadMessageService;
@@ -141,6 +142,13 @@ public OpenAI.Moderations moderations() {
141142
throw new UnsupportedOperationException(NOT_IMPLEMENTED);
142143
}
143144

145+
/**
146+
* Throw not implemented
147+
*/
148+
public OpenAI.Uploads uploads() {
149+
throw new UnsupportedOperationException(NOT_IMPLEMENTED);
150+
}
151+
144152
/**
145153
* Throw not implemented
146154
*/

src/main/java/io/github/sashirestela/openai/OpenAI.java

+57
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,11 @@
4343
import io.github.sashirestela.openai.domain.model.Model;
4444
import io.github.sashirestela.openai.domain.moderation.Moderation;
4545
import io.github.sashirestela.openai.domain.moderation.ModerationRequest;
46+
import io.github.sashirestela.openai.domain.upload.Upload;
47+
import io.github.sashirestela.openai.domain.upload.UploadCompleteRequest;
48+
import io.github.sashirestela.openai.domain.upload.UploadPart;
49+
import io.github.sashirestela.openai.domain.upload.UploadPartRequest;
50+
import io.github.sashirestela.openai.domain.upload.UploadRequest;
4651

4752
import java.io.InputStream;
4853
import java.util.EnumSet;
@@ -718,6 +723,58 @@ interface Moderations {
718723

719724
}
720725

726+
/**
727+
* Allows you to upload large files in multiple parts.
728+
*
729+
* @see <a href= "https://platform.openai.com/docs/api-reference/uploads">OpenAI Upload</a>
730+
*/
731+
@Resource("/v1/uploads")
732+
interface Uploads {
733+
734+
/**
735+
* Creates an intermediate Upload object that you can add Parts to.
736+
*
737+
* @param uploadRequest The creation request.
738+
* @return The Upload object with status pending.
739+
*/
740+
@POST
741+
CompletableFuture<Upload> create(@Body UploadRequest uploadRequest);
742+
743+
/**
744+
* Adds a Part to an Upload object. A Part represents a chunk of bytes from the file you are trying
745+
* to upload.
746+
*
747+
* @param uploadId The ID of the Upload.
748+
* @param uploadPartRequest The chunk of bytes for this Part.
749+
* @return The upload Part object.
750+
*/
751+
@Multipart
752+
@POST("/{uploadId}/parts")
753+
CompletableFuture<UploadPart> addPart(@Path("uploadId") String uploadId,
754+
@Body UploadPartRequest uploadPartRequest);
755+
756+
/**
757+
* Completes the Upload.
758+
*
759+
* @param uploadId The ID of the Upload.
760+
* @param uploadCompleteRequest The request including the ordered list of Part IDs.
761+
* @return The Upload object with status completed.
762+
*/
763+
@POST("/{uploadId}/complete")
764+
CompletableFuture<Upload> complete(@Path("uploadId") String uploadId,
765+
@Body UploadCompleteRequest uploadCompleteRequest);
766+
767+
/**
768+
* Cancels the Upload. No Parts may be added after an Upload is cancelled.
769+
*
770+
* @param uploadId The ID of the Upload.
771+
* @return The Upload object with status cancelled.
772+
*/
773+
@POST("/{uploadId}/cancel")
774+
CompletableFuture<Upload> cancel(@Path("uploadId") String uploadId);
775+
776+
}
777+
721778
static AudioResponseFormat getResponseFormat(AudioResponseFormat currValue, AudioResponseFormat orDefault,
722779
String methodName) {
723780
final var jsonEnumSet = EnumSet.of(AudioResponseFormat.JSON, AudioResponseFormat.VERBOSE_JSON);

src/main/java/io/github/sashirestela/openai/SimpleOpenAI.java

+13
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,19 @@ public OpenAI.Moderations moderations() {
152152
return moderationService;
153153
}
154154

155+
/**
156+
* Generates an implementation of the Uploads interface to handle requests.
157+
*
158+
* @return An instance of the interface.
159+
*/
160+
@Override
161+
public OpenAI.Uploads uploads() {
162+
if (uploadService == null) {
163+
uploadService = cleverClient.create(OpenAI.Uploads.class);
164+
}
165+
return uploadService;
166+
}
167+
155168
@Override
156169
public OpenAIBeta2.Assistants assistants() {
157170
if (assistantService == null) {
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
package io.github.sashirestela.openai.domain.upload;
2+
3+
import com.fasterxml.jackson.annotation.JsonProperty;
4+
import com.fasterxml.jackson.databind.PropertyNamingStrategies;
5+
import com.fasterxml.jackson.databind.annotation.JsonNaming;
6+
import io.github.sashirestela.openai.domain.file.FileResponse;
7+
import lombok.Getter;
8+
import lombok.NoArgsConstructor;
9+
import lombok.ToString;
10+
11+
@NoArgsConstructor
12+
@Getter
13+
@ToString
14+
@JsonNaming(PropertyNamingStrategies.SnakeCaseStrategy.class)
15+
public class Upload {
16+
17+
private String id;
18+
private Long createdAt;
19+
private String filename;
20+
private Long bytes;
21+
private String purpose;
22+
private UploadStatus status;
23+
private Long expiresAt;
24+
private String object;
25+
private FileResponse file;
26+
27+
public enum UploadStatus {
28+
29+
@JsonProperty("pending")
30+
PENDING,
31+
32+
@JsonProperty("completed")
33+
COMPLETED,
34+
35+
@JsonProperty("cancelled")
36+
CANCELLED,
37+
38+
@JsonProperty("expired")
39+
EXPIRED;
40+
41+
}
42+
43+
}

0 commit comments

Comments
 (0)