Autocomplete context improvement for codestral and qwen coder

[ferenci84]

Description

Enhance context inclusion for codestral and qwen coder.

Changes:

1. RecentlyVisitedRangesService from disabled to enabled by default, and numSorroundingLines to 20
2. In core/autocomplete/templating/filtering.ts moved diff to the back to avoid fill up the prompt leaving no room for other context.
3. Created the following experimental options to allow contributors and tech-savvy users experiment with enable/disable and change the order of context before someone concludes to a working model:

  // true = enabled, false = disabled, number = enabled with priority
  experimental_includeClipboard: boolean | number;
  experimental_includeRecentlyVisitedRanges: boolean | number;
  experimental_includeRecentlyEditedRanges: boolean | number;
  experimental_includeDiff: boolean | number;

4. Changed the templates for codestral and Qwen coder to properly include the context in the prompt
5. Added an interface AutocompleteCompletionOptions (with only one key, more can be added if needed) to provide additional information specific for the model that can be used when sending the request to the provider. In this case I added promptOnly parameter to let ollama provider know that the suffix shouldn't be sent as a separate key. This avoids ollama applying it's own fim template as we already apply a template before sending the prompt to ollama.

Relates to these issues:
#3900 (opened by me as a feature request, however I realized that the feature is already there just not working)
#2501
#1952
#3506

(most likely there are more)

Checklist

• The relevant docs, if any, have been updated or created
• The relevant tests, if any, have been updated or created

Testing instructions

1. Run the extension and open the Extension Development Host
2. Close the bottom panel if it's open (do not open the "Output" tab before the end of the test)
3. Set up codestral with remote provider and/or qwen coder with ollama as per the instructions of Continue documentation
4. Add this code into the css test file:

.class1 {
  background-color: red;
}

.class1 .myclass2 { /* I would visit this section last */
  background-color: blue;
}

Add this code to the html test file:

<div class="class1">
    <div class=></div>
  </div>

5. Run "Developer: Reload Window"
6. Focus on the css test file somewhere around myclass2
7. Focus on the html test file after class= and press "
8. It should autocomplete myclass2
9. Now you can open the bottom panel and the Output tab, and check that the most recently visited context (that is the css file) is added just before the prefix (<|fim_prefix|> for qwen and [PREFIX] for codestral.

Note: codestral tends to complete till the end of the file (which is magically handled in the postprocessing).

Possible further improvements on this feature:

• Some code snippets are duplicated. Some logic can be built to resolve overlapping snippets.
• recentlyVisitedRanges also contain contents from other windows like terminal or output if they are visible. We should handle them separately so that we can control their priority and whether they should be included or not.
• Limit diff in case there are a lot of changes
• Test other providers and models

[netlify]

✅ Deploy Preview for continuedev canceled.

Name	Link
🔨 Latest commit	5aaab9c
🔍 Latest deploy log	https://app.netlify.com/sites/continuedev/deploys/679f10d48af624000884e9bc