-
-
Notifications
You must be signed in to change notification settings - Fork 461
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(StreamsExtractor): generate PoToken #7069
Draft
FineFindus
wants to merge
6
commits into
libre-tube:master
Choose a base branch
from
FineFindus:feat/potoken
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+697
−12
Draft
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
8771dc4
feat(StreamsExtractor): generate PoToken
FineFindus bb4251c
refactor(PoTokenWebView): use `USER_AGENT` from ExternalApi
FineFindus 2feb3a1
feat: use latest NewPipe implementation
FineFindus 2077519
refactor: move local stream extraction to sub-package
FineFindus f757be0
refactor: use kotlinx.json
FineFindus 6ecad37
feat(Preferences): disable HLS by default
FineFindus File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,127 @@ | ||
<!DOCTYPE html> | ||
<html lang="en"><head><title></title><script> | ||
/** | ||
* Factory method to create and load a BotGuardClient instance. | ||
* @param options - Configuration options for the BotGuardClient. | ||
* @returns A promise that resolves to a loaded BotGuardClient instance. | ||
*/ | ||
function loadBotGuard(challengeData) { | ||
this.vm = this[challengeData.globalName]; | ||
this.program = challengeData.program; | ||
this.vmFunctions = {}; | ||
this.syncSnapshotFunction = null; | ||
|
||
if (!this.vm) | ||
throw new Error('[BotGuardClient]: VM not found in the global object'); | ||
|
||
if (!this.vm.a) | ||
throw new Error('[BotGuardClient]: Could not load program'); | ||
|
||
const vmFunctionsCallback = function ( | ||
asyncSnapshotFunction, | ||
shutdownFunction, | ||
passEventFunction, | ||
checkCameraFunction | ||
) { | ||
this.vmFunctions = { | ||
asyncSnapshotFunction: asyncSnapshotFunction, | ||
shutdownFunction: shutdownFunction, | ||
passEventFunction: passEventFunction, | ||
checkCameraFunction: checkCameraFunction | ||
}; | ||
}; | ||
|
||
this.syncSnapshotFunction = this.vm.a(this.program, vmFunctionsCallback, true, this.userInteractionElement, function () {/** no-op */ }, [ [], [] ])[0] | ||
|
||
// an asynchronous function runs in the background and it will eventually call | ||
// `vmFunctionsCallback`, however we need to manually tell JavaScript to pass | ||
// control to the things running in the background by interrupting this async | ||
// function in any way, e.g. with a delay of 1ms. The loop is most probably not | ||
// needed but is there just because. | ||
return new Promise(function (resolve, reject) { | ||
i = 0 | ||
refreshIntervalId = setInterval(function () { | ||
if (!!this.vmFunctions.asyncSnapshotFunction) { | ||
resolve(this) | ||
clearInterval(refreshIntervalId); | ||
} | ||
if (i >= 10000) { | ||
reject("asyncSnapshotFunction is null even after 10 seconds") | ||
clearInterval(refreshIntervalId); | ||
} | ||
i += 1; | ||
}, 1); | ||
}) | ||
} | ||
|
||
/** | ||
* Takes a snapshot asynchronously. | ||
* @returns The snapshot result. | ||
* @example | ||
* ```ts | ||
* const result = await botguard.snapshot({ | ||
* contentBinding: { | ||
* c: "a=6&a2=10&b=SZWDwKVIuixOp7Y4euGTgwckbJA&c=1729143849&d=1&t=7200&c1a=1&c6a=1&c6b=1&hh=HrMb5mRWTyxGJphDr0nW2Oxonh0_wl2BDqWuLHyeKLo", | ||
* e: "ENGAGEMENT_TYPE_VIDEO_LIKE", | ||
* encryptedVideoId: "P-vC09ZJcnM" | ||
* } | ||
* }); | ||
* | ||
* console.log(result); | ||
* ``` | ||
*/ | ||
function snapshot(args) { | ||
return new Promise(function (resolve, reject) { | ||
if (!this.vmFunctions.asyncSnapshotFunction) | ||
return reject(new Error('[BotGuardClient]: Async snapshot function not found')); | ||
|
||
this.vmFunctions.asyncSnapshotFunction(function (response) { resolve(response) }, [ | ||
args.contentBinding, | ||
args.signedTimestamp, | ||
args.webPoSignalOutput, | ||
args.skipPrivacyBuffer | ||
]); | ||
}); | ||
} | ||
|
||
function runBotGuard(challengeData) { | ||
const interpreterJavascript = challengeData.interpreterJavascript.privateDoNotAccessOrElseSafeScriptWrappedValue; | ||
|
||
if (interpreterJavascript) { | ||
new Function(interpreterJavascript)(); | ||
} else throw new Error('Could not load VM'); | ||
|
||
const webPoSignalOutput = []; | ||
return loadBotGuard({ | ||
globalName: challengeData.globalName, | ||
globalObj: this, | ||
program: challengeData.program | ||
}).then(function (botguard) { | ||
return botguard.snapshot({ webPoSignalOutput: webPoSignalOutput }) | ||
}).then(function (botguardResponse) { | ||
return { webPoSignalOutput: webPoSignalOutput, botguardResponse: botguardResponse } | ||
}) | ||
} | ||
|
||
function obtainPoToken(webPoSignalOutput, integrityToken, identifier) { | ||
const getMinter = webPoSignalOutput[0]; | ||
|
||
if (!getMinter) | ||
throw new Error('PMD:Undefined'); | ||
|
||
const mintCallback = getMinter(integrityToken); | ||
|
||
if (!(mintCallback instanceof Function)) | ||
throw new Error('APF:Failed'); | ||
|
||
const result = mintCallback(identifier); | ||
|
||
if (!result) | ||
throw new Error('YNJ:Undefined'); | ||
|
||
if (!(result instanceof Uint8Array)) | ||
throw new Error('ODM:Invalid'); | ||
|
||
return result; | ||
} | ||
</script></head><body></body></html> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
133 changes: 133 additions & 0 deletions
133
app/src/main/java/com/github/libretube/api/local/JavaScriptUtil.kt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,133 @@ | ||
package com.github.libretube.api.local | ||
|
||
import okio.ByteString.Companion.decodeBase64 | ||
import okio.ByteString.Companion.toByteString | ||
|
||
import kotlinx.serialization.json.Json | ||
import kotlinx.serialization.json.JsonNull | ||
import kotlinx.serialization.json.JsonObject | ||
import kotlinx.serialization.json.JsonPrimitive | ||
import kotlinx.serialization.json.jsonArray | ||
import kotlinx.serialization.json.jsonPrimitive | ||
import kotlinx.serialization.json.long | ||
|
||
/** | ||
* Parses the raw challenge data obtained from the Create endpoint and returns an object that can be | ||
* embedded in a JavaScript snippet. | ||
*/ | ||
fun parseChallengeData(rawChallengeData: String): String { | ||
val scrambled = Json.parseToJsonElement(rawChallengeData).jsonArray | ||
|
||
val challengeData = if (scrambled.size > 1 && scrambled[1].jsonPrimitive.isString) { | ||
val descrambled = descramble(scrambled[1].jsonPrimitive.content) | ||
Json.parseToJsonElement(descrambled).jsonArray | ||
} else { | ||
scrambled[1].jsonArray | ||
} | ||
|
||
val messageId = challengeData[0].jsonPrimitive.content | ||
val interpreterHash = challengeData[3].jsonPrimitive.content | ||
val program = challengeData[4].jsonPrimitive.content | ||
val globalName = challengeData[5].jsonPrimitive.content | ||
val clientExperimentsStateBlob = challengeData[7].jsonPrimitive.content | ||
|
||
|
||
val privateDoNotAccessOrElseSafeScriptWrappedValue = challengeData[1] | ||
.takeIf { it !is JsonNull } | ||
?.jsonArray | ||
?.find { it.jsonPrimitive.isString } | ||
|
||
val privateDoNotAccessOrElseTrustedResourceUrlWrappedValue = challengeData[2] | ||
.takeIf { it !is JsonNull } | ||
?.jsonArray | ||
?.find { it.jsonPrimitive.isString } | ||
|
||
|
||
return Json.encodeToString( | ||
JsonObject.serializer(), JsonObject( | ||
mapOf( | ||
"messageId" to JsonPrimitive(messageId), | ||
"interpreterJavascript" to JsonObject( | ||
mapOf( | ||
"privateDoNotAccessOrElseSafeScriptWrappedValue" to (privateDoNotAccessOrElseSafeScriptWrappedValue | ||
?: JsonPrimitive("")), | ||
"privateDoNotAccessOrElseTrustedResourceUrlWrappedValue" to (privateDoNotAccessOrElseTrustedResourceUrlWrappedValue | ||
?: JsonPrimitive("")) | ||
) | ||
), | ||
"interpreterHash" to JsonPrimitive(interpreterHash), | ||
"program" to JsonPrimitive(program), | ||
"globalName" to JsonPrimitive(globalName), | ||
"clientExperimentsStateBlob" to JsonPrimitive(clientExperimentsStateBlob) | ||
) | ||
) | ||
) | ||
} | ||
|
||
/** | ||
* Parses the raw integrity token data obtained from the GenerateIT endpoint to a JavaScript | ||
* `Uint8Array` that can be embedded directly in JavaScript code, and an [Int] representing the | ||
* duration of this token in seconds. | ||
*/ | ||
fun parseIntegrityTokenData(rawIntegrityTokenData: String): Pair<String, Long> { | ||
val integrityTokenData = Json.parseToJsonElement(rawIntegrityTokenData).jsonArray | ||
return base64ToU8(integrityTokenData[0].jsonPrimitive.content) to integrityTokenData[1].jsonPrimitive.long | ||
} | ||
|
||
/** | ||
* Converts a string (usually the identifier used as input to `obtainPoToken`) to a JavaScript | ||
* `Uint8Array` that can be embedded directly in JavaScript code. | ||
*/ | ||
fun stringToU8(identifier: String): String { | ||
return newUint8Array(identifier.toByteArray()) | ||
} | ||
|
||
/** | ||
* Takes a poToken encoded as a sequence of bytes represented as integers separated by commas | ||
* (e.g. "97,98,99" would be "abc"), which is the output of `Uint8Array::toString()` in JavaScript, | ||
* and converts it to the specific base64 representation for poTokens. | ||
*/ | ||
fun u8ToBase64(poToken: String): String { | ||
return poToken.split(",") | ||
.map { it.toUByte().toByte() } | ||
.toByteArray() | ||
.toByteString() | ||
.base64() | ||
.replace("+", "-") | ||
.replace("/", "_") | ||
} | ||
|
||
/** | ||
* Takes the scrambled challenge, decodes it from base64, adds 97 to each byte. | ||
*/ | ||
private fun descramble(scrambledChallenge: String): String { | ||
return base64ToByteString(scrambledChallenge) | ||
.map { (it + 97).toByte() } | ||
.toByteArray() | ||
.decodeToString() | ||
} | ||
|
||
/** | ||
* Decodes a base64 string encoded in the specific base64 representation used by YouTube, and | ||
* returns a JavaScript `Uint8Array` that can be embedded directly in JavaScript code. | ||
*/ | ||
private fun base64ToU8(base64: String): String { | ||
return newUint8Array(base64ToByteString(base64)) | ||
} | ||
|
||
private fun newUint8Array(contents: ByteArray): String { | ||
return "new Uint8Array([" + contents.joinToString(separator = ",") { it.toUByte().toString() } + "])" | ||
} | ||
|
||
/** | ||
* Decodes a base64 string encoded in the specific base64 representation used by YouTube. | ||
*/ | ||
private fun base64ToByteString(base64: String): ByteArray { | ||
val base64Mod = base64 | ||
.replace('-', '+') | ||
.replace('_', '/') | ||
.replace('.', '=') | ||
|
||
return (base64Mod.decodeBase64() ?: throw PoTokenException("Cannot base64 decode")) | ||
.toByteArray() | ||
} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use
JsonNull
instead ofJsonPrimitive("")
to make it consistentthe other values (like
JsonPrimitive(messageId)
) are also returned asJsonNull
if they are nullfrom nanojson
value
function:so I guess
JsonNull
better matches the previous/NewPipe code