Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add size for each report and result #42

Merged
merged 2 commits into from
Dec 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions app/components/reports-table.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ const columns = [
{ name: 'ID', uid: 'reportID' },
{ name: 'Project', uid: 'project' },
{ name: 'Created At', uid: 'createdAt' },
{ name: 'Size', uid: 'size' },
{ name: 'Actions', uid: 'actions' },
];

Expand Down Expand Up @@ -144,6 +145,7 @@ export default function ReportsTable({ onChange }: ReportsTableProps) {
<TableCell className="w-1/4">
<FormattedDate date={item.createdAt} />
</TableCell>
<TableCell className="w-1/4">{item.size}</TableCell>
<TableCell className="w-1/4">
<div className="flex gap-4 justify-end">
<Tooltip color="success" content="Open Report" placement="top">
Expand Down
4 changes: 3 additions & 1 deletion app/components/results-table.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,12 @@ const columns = [
{ name: 'Project', uid: 'project' },
{ name: 'Created At', uid: 'createdAt' },
{ name: 'Tags', uid: 'tags' },
{ name: 'Size', uid: 'size' },
{ name: 'Actions', uid: 'actions' },
];

const getTags = (item: Result) => {
return Object.entries(item).filter(([key]) => !['resultID', 'createdAt', 'project'].includes(key));
return Object.entries(item).filter(([key]) => !['resultID', 'createdAt', 'size', 'project'].includes(key));
};

interface ResultsTableProps {
Expand Down Expand Up @@ -172,6 +173,7 @@ export default function ResultsTable({ onSelect, onDeleted, selected }: ResultsT
>{`${key}: ${value}`}</Chip>
))}
</TableCell>
<TableCell className="w-1/4">{item.size}</TableCell>
<TableCell className="w-1/12">
<div className="flex gap-4 justify-center">
<DeleteResultsButton resultIds={[item.resultID]} onDeletedResult={shouldRefetch} />
Expand Down
15 changes: 15 additions & 0 deletions app/lib/storage/file.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { REPORTS_BUCKET } from './constants';

export const isUUID = (uuid?: string): boolean => {
return /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(uuid ?? '');
};

export const getFileReportID = (filePath: string): string => {
const parts = filePath.split(REPORTS_BUCKET).pop()?.split('/') ?? [];

const noProjectName = isUUID(parts?.at(1));

const reportIdIndex = noProjectName ? 1 : 2;

return parts?.at(reportIdIndex) ?? '';
};
11 changes: 9 additions & 2 deletions app/lib/storage/fs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
await fs.access(dir);
} catch {
await fs.mkdir(dir, { recursive: true });
console.log('Created directory:', dir);

Check warning on line 32 in app/lib/storage/fs.ts

View workflow job for this annotation

GitHub Actions / eslint

Unexpected console statement
}
}

Expand Down Expand Up @@ -148,14 +148,17 @@
const reports = await Promise.all(
paginatedFiles.map(async (file) => {
const id = path.basename(file.filePath);
const parentDir = path.basename(path.dirname(file.filePath));
const reportPath = path.dirname(file.filePath);
const parentDir = path.basename(reportPath);
const size = await getFolderSizeInMb(path.join(reportPath, id));

const projectName = parentDir === REPORTS_PATH ? '' : parentDir;

return {
reportID: id,
project: projectName,
createdAt: file.birthtime,
size,
reportUrl: `${serveReportRoute}/${projectName ? encodeURIComponent(projectName) : ''}/${id}/index.html`,
};
}),
Expand Down Expand Up @@ -194,8 +197,11 @@
export async function saveResult(buffer: Buffer, resultDetails: ResultDetails) {
await createDirectoriesIfMissing();
const resultID = randomUUID();
const resultPath = path.join(RESULTS_FOLDER, `${resultID}.zip`);

const { error: writeZipError } = await withError(fs.writeFile(resultPath, buffer));

const { error: writeZipError } = await withError(fs.writeFile(path.join(RESULTS_FOLDER, `${resultID}.zip`), buffer));
const size = await getFolderSizeInMb(resultPath);

if (writeZipError) {
throw new Error(`failed to save result ${resultID} zip file: ${writeZipError.message}`);
Expand All @@ -204,6 +210,7 @@
const metaData = {
resultID,
createdAt: new Date().toISOString(),
size,
...resultDetails,
};

Expand Down Expand Up @@ -239,7 +246,7 @@
const { error } = await withError(fs.rm(tempFolder, { recursive: true, force: true }));

if (error) {
console.log(`failed to remove temp folder: ${error.message}`);

Check warning on line 249 in app/lib/storage/fs.ts

View workflow job for this annotation

GitHub Actions / eslint

Unexpected console statement
}

return reportId;
Expand Down
22 changes: 20 additions & 2 deletions app/lib/storage/s3.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@
import { generatePlaywrightReport } from '@/app/lib/pw';
import { withError } from '@/app/lib/withError';
import { type Result, type Report, type ResultDetails, type ServerDataInfo } from '@/app/lib/storage/types';
import { env } from '@/app/config/env';

Check warning on line 17 in app/lib/storage/s3.ts

View workflow job for this annotation

GitHub Actions / eslint

There should be at least one empty line between import groups
import { getFileReportID } from './file';

Check warning on line 18 in app/lib/storage/s3.ts

View workflow job for this annotation

GitHub Actions / eslint

`./file` import should occur before import of `@/app/lib/constants`

const createClient = () => {
const endPoint = env.S3_ENDPOINT;
Expand Down Expand Up @@ -279,12 +280,20 @@
const reportsStream = this.client.listObjectsV2(this.bucket, REPORTS_BUCKET, true);

const reports: Report[] = [];
const reportSizes = new Map<string, number>();

return new Promise((resolve, reject) => {
reportsStream.on('data', (file) => {
if (!file?.name) {
return;
}

const reportID = getFileReportID(file.name);

const newSize = (reportSizes.get(reportID) ?? 0) + file.size;

reportSizes.set(reportID, newSize);

if (!file.name.endsWith('index.html') || file.name.includes('trace')) {
return;
}
Expand All @@ -308,6 +317,7 @@
project: projectName,
createdAt: file.lastModified,
reportUrl: `${serveReportRoute}/${projectName ? encodeURIComponent(projectName) : ''}/${id}/index.html`,
size: '',
};

if (noFilters || shouldFilterByProject || shouldFilterByID) {
Expand All @@ -326,7 +336,13 @@

const currentReports = handlePagination<Report>(reports, input?.pagination);

resolve({ reports: currentReports, total: reports.length });
resolve({
reports: currentReports.map((report) => ({
...report,
size: bytesToString(reportSizes.get(report.reportID) ?? 0),
})),
total: reports.length,
});
});
});
}
Expand Down Expand Up @@ -371,11 +387,13 @@
await withError(this.clear(...objects));
}

async saveResult(buffer: Buffer, resultDetails: ResultDetails): Promise<{ resultID: UUID; createdAt: string }> {
async saveResult(buffer: Buffer, resultDetails: ResultDetails) {
const resultID = randomUUID();
const size = bytesToString(buffer.length);

const metaData = {
resultID,
size,
createdAt: new Date().toISOString(),
...resultDetails,
};
Expand Down
10 changes: 9 additions & 1 deletion app/lib/storage/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ export interface Storage {
) => Promise<{
resultID: UUID;
createdAt: string;
size: string;
}>;
generateReport: (resultsIds: string[], project?: string) => Promise<UUID>;
getReportsProjects: () => Promise<string[]>;
Expand Down Expand Up @@ -53,9 +54,16 @@ export type Result = {
resultID: UUID;
createdAt: string;
project: string;
size: string;
} & ResultDetails;

export type Report = { reportID: string; project: string; reportUrl: string; createdAt: Date };
export type Report = {
reportID: string;
project: string;
reportUrl: string;
createdAt: Date;
size: string;
};

export type ReportHistory = Report & ReportInfo;

Expand Down
10 changes: 7 additions & 3 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Check out the live demo: [familiar-alyss-alex-hot-6926ec9c.koyeb.app](https://fa
Returns list of generated reports and corresponding url on server:

```sh
curl --location --request DELETE 'http://localhost:3000/api/report/delete' \
curl --location --request GET 'http://localhost:3000/api/report/list' \
--header 'Content-Type: application/json' \
--header 'Authorization: <api-token>' \
--data '{
Expand All @@ -84,12 +84,14 @@ Response example:
"reportID": "8e9af87d-1d10-4729-aefd-3e92ee64d06c",
"createdAt": "2024-05-06T16:52:45.017Z",
"project": "regression",
"size": "6.97 MB",
"reportUrl": "/api/serve/regression/8e9af87d-1d10-4729-aefd-3e92ee64d06c/index.html"
},
{
"reportID": "8fe427ed-783c-4fb9-aacc-ba6fbc5f5667",
"createdAt": "2024-05-06T16:59:38.814Z",
"project": "smoke",
"size": "1.53 MB",
"reportUrl": "/api/serve/smoke/8fe427ed-783c-4fb9-aacc-ba6fbc5f5667/index.html"
}
],
Expand Down Expand Up @@ -126,7 +128,7 @@ Response example:
Generates report from provided resultsIds, merges results together into one report, using https://playwright.dev/docs/test-sharding#merge-reports-cli

```sh
curl --location 'http://localhost:3000/api/report/generate' \
curl --location --request POST 'http://localhost:3000/api/report/generate' \
--header 'Content-Type: application/json' \
--header 'Authorization: <api-token>' \
--data '{
Expand Down Expand Up @@ -164,6 +166,7 @@ Response will contain array of results:
{
"resultID": "a7beb04b-f190-4fbb-bebd-58b2c776e6c3",
"createdAt": "2024-05-06T16:40:33.021Z",
"size": "1.93 MB",
"project": "regression",
"testRunName": "regression-run-v1.10",
"reporter": "okhotemskyi"
Expand Down Expand Up @@ -204,7 +207,8 @@ Response example:
"createdAt": "2024-07-07T13:35:57.382Z",
"project": "desktop",
"reporter": "okhotemskyi",
"appVersion": "1.2.2"
"appVersion": "1.2.2",
"size": "1.2 MB"
},
"status": 201
}
Expand Down
Loading