Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(logging): Add interactionId to header of Cody Client requests for autoCompletes and autoEdits #6749

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions vscode/src/autoedits/adapters/base.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,5 @@ export interface AutoeditModelOptions {
codeToRewrite: string
userId: string | null
isChatModel: boolean
requestId: string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's name it interactionId for clarity. The requestId refers to the analytics logger item ID which represents the current state of the auto-edit request.

}
1 change: 1 addition & 0 deletions vscode/src/autoedits/adapters/cody-gateway.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
codeToRewrite: 'const x = 1',
userId: 'test-user',
isChatModel: true,
requestId: 'test-request-id',
}

const mockFetch = vi.fn()
Expand Down Expand Up @@ -56,7 +57,7 @@
await adapter.getModelResponse(options)

// Verify the fetch call
expect(mockFetch).toHaveBeenCalledWith(options.url, {

Check failure on line 60 in vscode/src/autoedits/adapters/cody-gateway.test.ts

View workflow job for this annotation

GitHub Actions / test-unit (ubuntu, 20)

src/autoedits/adapters/cody-gateway.test.ts > CodyGatewayAdapter > sends correct request parameters for chat model

AssertionError: expected "spy" to be called with arguments: [ …(2) ] Received: 1st spy call: Array [ "https://test-gateway.sourcegraph.com/v1/completions", Object { - "body": StringContaining "\"model\":\"anthropic/claude-2\"", + "body": "{\"stream\":false,\"model\":\"anthropic/claude-2\",\"temperature\":0,\"max_tokens\":515,\"response_format\":{\"type\":\"text\"},\"prediction\":{\"type\":\"content\",\"content\":\"const x = 1\"},\"rewrite_speculation\":true,\"user\":\"test-user\",\"messages\":[{\"role\":\"system\",\"content\":\"system message\"},{\"role\":\"user\",\"content\":\"user message\"}]}", "headers": Object { - "Authorization": StringContaining "sgd_", + "Authorization": "***", "Content-Type": "application/json", "X-Sourcegraph-Feature": "code_completions", + "X-Sourcegraph-Interaction-ID": "test-request-id", }, "method": "POST", }, ] Number of calls: 1 ❯ src/autoedits/adapters/cody-gateway.test.ts:60:27

Check failure on line 60 in vscode/src/autoedits/adapters/cody-gateway.test.ts

View workflow job for this annotation

GitHub Actions / test-unit (windows, 20)

src/autoedits/adapters/cody-gateway.test.ts > CodyGatewayAdapter > sends correct request parameters for chat model

AssertionError: expected "spy" to be called with arguments: [ …(2) ] Received: 1st spy call: Array [ "https://test-gateway.sourcegraph.com/v1/completions", Object { - "body": StringContaining "\"model\":\"anthropic/claude-2\"", + "body": "{\"stream\":false,\"model\":\"anthropic/claude-2\",\"temperature\":0,\"max_tokens\":515,\"response_format\":{\"type\":\"text\"},\"prediction\":{\"type\":\"content\",\"content\":\"const x = 1\"},\"rewrite_speculation\":true,\"user\":\"test-user\",\"messages\":[{\"role\":\"system\",\"content\":\"system message\"},{\"role\":\"user\",\"content\":\"user message\"}]}", "headers": Object { - "Authorization": StringContaining "sgd_", + "Authorization": "***", "Content-Type": "application/json", "X-Sourcegraph-Feature": "code_completions", + "X-Sourcegraph-Interaction-ID": "test-request-id", }, "method": "POST", }, ] Number of calls: 1 ❯ src/autoedits/adapters/cody-gateway.test.ts:60:27

Check failure on line 60 in vscode/src/autoedits/adapters/cody-gateway.test.ts

View workflow job for this annotation

GitHub Actions / test-unit (ubuntu, 18)

src/autoedits/adapters/cody-gateway.test.ts > CodyGatewayAdapter > sends correct request parameters for chat model

AssertionError: expected "spy" to be called with arguments: [ …(2) ] Received: 1st spy call: Array [ "https://test-gateway.sourcegraph.com/v1/completions", Object { - "body": StringContaining "\"model\":\"anthropic/claude-2\"", + "body": "{\"stream\":false,\"model\":\"anthropic/claude-2\",\"temperature\":0,\"max_tokens\":515,\"response_format\":{\"type\":\"text\"},\"prediction\":{\"type\":\"content\",\"content\":\"const x = 1\"},\"rewrite_speculation\":true,\"user\":\"test-user\",\"messages\":[{\"role\":\"system\",\"content\":\"system message\"},{\"role\":\"user\",\"content\":\"user message\"}]}", "headers": Object { - "Authorization": StringContaining "sgd_", + "Authorization": "***", "Content-Type": "application/json", "X-Sourcegraph-Feature": "code_completions", + "X-Sourcegraph-Interaction-ID": "test-request-id", }, "method": "POST", }, ] Number of calls: 1 ❯ src/autoedits/adapters/cody-gateway.test.ts:60:27
method: 'POST',
headers: {
'Content-Type': 'application/json',
Expand Down
1 change: 1 addition & 0 deletions vscode/src/autoedits/adapters/cody-gateway.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ export class CodyGatewayAdapter implements AutoeditsModelAdapter {
public async getModelResponse(options: AutoeditModelOptions): Promise<string> {
const headers = {
'X-Sourcegraph-Feature': 'code_completions',
'X-Sourcegraph-Interaction-ID': options.requestId,
}
const body = this.getMessageBody(options)
try {
Expand Down
1 change: 1 addition & 0 deletions vscode/src/autoedits/adapters/fireworks.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
codeToRewrite: 'const x = 1',
userId: 'test-user',
isChatModel: true,
requestId: 'test-request-id',
}

const apiKey = 'test-api-key'
Expand Down Expand Up @@ -50,7 +51,7 @@

await adapter.getModelResponse(options)

expect(mockFetch).toHaveBeenCalledWith(options.url, {

Check failure on line 54 in vscode/src/autoedits/adapters/fireworks.test.ts

View workflow job for this annotation

GitHub Actions / test-unit (ubuntu, 20)

src/autoedits/adapters/fireworks.test.ts > FireworksAdapter > sends correct request parameters for chat model

AssertionError: expected "spy" to be called with arguments: [ …(2) ] Received: 1st spy call: Array [ "https://api.fireworks.ai/v1/completions", Object { - "body": StringContaining "\"model\":\"accounts/fireworks/models/llama-v2-7b\"", + "body": "{\"stream\":false,\"model\":\"accounts/fireworks/models/llama-v2-7b\",\"temperature\":0,\"max_tokens\":515,\"response_format\":{\"type\":\"text\"},\"prediction\":{\"type\":\"content\",\"content\":\"const x = 1\"},\"user\":\"test-user\",\"messages\":[{\"role\":\"system\",\"content\":\"system message\"},{\"role\":\"user\",\"content\":\"user message\"}]}", "headers": Object { "Authorization": "***", "Content-Type": "application/json", + "X-Sourcegraph-Interaction-ID": "test-request-id", }, "method": "POST", }, ] Number of calls: 1 ❯ src/autoedits/adapters/fireworks.test.ts:54:27

Check failure on line 54 in vscode/src/autoedits/adapters/fireworks.test.ts

View workflow job for this annotation

GitHub Actions / test-unit (windows, 20)

src/autoedits/adapters/fireworks.test.ts > FireworksAdapter > sends correct request parameters for chat model

AssertionError: expected "spy" to be called with arguments: [ …(2) ] Received: 1st spy call: Array [ "https://api.fireworks.ai/v1/completions", Object { - "body": StringContaining "\"model\":\"accounts/fireworks/models/llama-v2-7b\"", + "body": "{\"stream\":false,\"model\":\"accounts/fireworks/models/llama-v2-7b\",\"temperature\":0,\"max_tokens\":515,\"response_format\":{\"type\":\"text\"},\"prediction\":{\"type\":\"content\",\"content\":\"const x = 1\"},\"user\":\"test-user\",\"messages\":[{\"role\":\"system\",\"content\":\"system message\"},{\"role\":\"user\",\"content\":\"user message\"}]}", "headers": Object { "Authorization": "***", "Content-Type": "application/json", + "X-Sourcegraph-Interaction-ID": "test-request-id", }, "method": "POST", }, ] Number of calls: 1 ❯ src/autoedits/adapters/fireworks.test.ts:54:27

Check failure on line 54 in vscode/src/autoedits/adapters/fireworks.test.ts

View workflow job for this annotation

GitHub Actions / test-unit (ubuntu, 18)

src/autoedits/adapters/fireworks.test.ts > FireworksAdapter > sends correct request parameters for chat model

AssertionError: expected "spy" to be called with arguments: [ …(2) ] Received: 1st spy call: Array [ "https://api.fireworks.ai/v1/completions", Object { - "body": StringContaining "\"model\":\"accounts/fireworks/models/llama-v2-7b\"", + "body": "{\"stream\":false,\"model\":\"accounts/fireworks/models/llama-v2-7b\",\"temperature\":0,\"max_tokens\":515,\"response_format\":{\"type\":\"text\"},\"prediction\":{\"type\":\"content\",\"content\":\"const x = 1\"},\"user\":\"test-user\",\"messages\":[{\"role\":\"system\",\"content\":\"system message\"},{\"role\":\"user\",\"content\":\"user message\"}]}", "headers": Object { "Authorization": "***", "Content-Type": "application/json", + "X-Sourcegraph-Interaction-ID": "test-request-id", }, "method": "POST", }, ] Number of calls: 1 ❯ src/autoedits/adapters/fireworks.test.ts:54:27
method: 'POST',
headers: {
'Content-Type': 'application/json',
Expand Down
5 changes: 4 additions & 1 deletion vscode/src/autoedits/adapters/fireworks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,10 @@ export class FireworksAdapter implements AutoeditsModelAdapter {
)
throw new Error('No api key provided in the config override')
}
const response = await getModelResponse(option.url, body, apiKey)
const headers = {
'X-Sourcegraph-Interaction-ID': option.requestId,
}
const response = await getModelResponse(option.url, body, apiKey, headers)
if (option.isChatModel) {
return response.choices[0].message.content
}
Expand Down
1 change: 1 addition & 0 deletions vscode/src/autoedits/adapters/sourcegraph-chat.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ describe('SourcegraphChatAdapter', () => {
codeToRewrite: 'const x = 1',
userId: 'test-user',
isChatModel: true,
requestId: 'test-request-id',
}

beforeEach(() => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ describe('SourcegraphCompletionsAdapter', () => {
codeToRewrite: 'const x = 1',
userId: 'test-user',
isChatModel: false,
requestId: 'test-request-id',
}

beforeEach(() => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ describe('AutoeditAnalyticsLogger', () => {
codeToRewrite: 'This is test code to rewrite',
userId: 'test-user-id',
isChatModel: false,
requestId: 'test-request-id',
}

function getRequestStartMetadata(): Parameters<AutoeditAnalyticsLogger['createRequest']>[0] {
Expand Down
4 changes: 4 additions & 0 deletions vscode/src/autoedits/autoedits-provider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,7 @@ export class AutoeditsProvider implements vscode.InlineCompletionItemProvider, v
position,
prompt,
codeToReplaceData,
requestId,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The requestId is different from the stableId we use as the interactionId for auto-edit events. The stableId is derived from the prediction text and is only created once we receive a response from the backend. So, adding this header to requests won’t let us link auto-edit events to the prediction request. Based on the Linear issue description, this might be fine, but I just wanted to flag it here.

})

if (abortSignal?.aborted) {
Expand Down Expand Up @@ -449,11 +450,13 @@ export class AutoeditsProvider implements vscode.InlineCompletionItemProvider, v
position,
codeToReplaceData,
prompt,
requestId,
}: {
document: vscode.TextDocument
position: vscode.Position
codeToReplaceData: CodeToReplaceData
prompt: AutoeditsPrompt
requestId: AutoeditRequestID
}): Promise<string | undefined> {
if (autoeditsProviderConfig.isMockResponseFromCurrentDocumentTemplateEnabled) {
const responseMetadata = extractAutoEditResponseFromCurrentDocumentCommentTemplate(
Expand All @@ -480,6 +483,7 @@ export class AutoeditsProvider implements vscode.InlineCompletionItemProvider, v
codeToRewrite: codeToReplaceData.codeToRewrite,
userId: (await currentResolvedConfig()).clientState.anonymousUserID,
isChatModel: autoeditsProviderConfig.isChatModel,
requestId: requestId,
})
}

Expand Down
1 change: 1 addition & 0 deletions vscode/src/completions/fast-path-client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ export function createFastPathClient(
headers.set('Content-Type', `application/json${fireworksConfig ? '' : '; charset=utf-8'}`)
headers.set('Authorization', `Bearer ${fastPathAccessToken}`)
headers.set('X-Sourcegraph-Feature', 'code_completions')
headers.set('X-Sourcegraph-Interaction-ID', providerOptions.completionLogId)
headers.set('X-Timeout-Ms', requestParams.timeoutMs.toString())
addTraceparent(headers)

Expand Down
5 changes: 3 additions & 2 deletions vscode/src/completions/providers/fireworks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -158,9 +158,10 @@ class FireworksProvider extends Provider {
})
}
}

const headers = this.getCustomHeaders(authStatus.isFireworksTracingEnabled)
headers['X-Sourcegraph-Interaction-ID'] = options.completionLogId
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move this logic inside the getCustomHeaders method? Then, this header will also propagate to the fast path client through the createFastPathClient call.

return this.client.complete(requestParams, abortController, {
customHeaders: this.getCustomHeaders(authStatus.isFireworksTracingEnabled),
customHeaders: headers,
})
}

Expand Down
Loading