-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path2024-11-03-readwise-and-github-actions.html
654 lines (568 loc) · 63.5 KB
/
2024-11-03-readwise-and-github-actions.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<link rel="alternate"
type="application/rss+xml"
href="https://www.vandee.art/rss.xml"
title="RSS feed for https://www.vandee.art/">
<title>Readwise 和 GitHub Actions 联用 - 流动知识检索</title>
<meta property="og:title" content="Readwise 和 GitHub Actions 联用 - 流动知识检索">
<meta property="og:type" content="article" />
<meta property="og:url" content="https://www.vandee.art/2024-11-03-readwise-and-github-actions.html">
<meta name="author" content="Vandee">
<meta name="referrer" content="no-referrer">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="stylesheet" href="assets/css/style.css" type="text/css"/>
<link rel="stylesheet"
href="https://lf26-cdn-tos.bytecdntp.com/cdn/expire-1-M/font-awesome/6.0.0/css/all.min.css"/>
<link rel="stylesheet"
href="https://testingcf.jsdelivr.net/npm/@fancyapps/[email protected]/dist/fancybox.css"/>
<link rel="icon" type="image/x-icon" href="/static/favicon.ico"/>
<script src="https://lf6-cdn-tos.bytecdntp.com/cdn/expire-1-M/jquery/3.6.0/jquery.min.js" defer></script>
<script src="https://testingcf.jsdelivr.net/npm/@fancyapps/[email protected]/dist/fancybox.umd.js" defer></script>
<script src="https://lf3-cdn-tos.bytecdntp.com/cdn/expire-1-M/pangu/4.0.7/pangu.min.js" defer></script>
<script defer>
document.addEventListener("DOMContentLoaded", function () {
pangu.spacingPage();
});
</script>
<script src="assets/js/app.js" defer></script>
<script src="assets/js/copyCode.js" defer></script>
<script src="assets/js/search.js" defer></script></head>
<body>
<div id="preamble" class="status">
<header>
<h1><a href="https://www.vandee.art/">Vandee's Blog</a></h1>
<nav>
<a href="https://www.vandee.art/">Home</a>
<a href="https://x.vandee.art/wiki">Wiki</a>
<a href="https://x.vandee.art/photo">Photo</a>
<a href="archive.html">Archive</a>
<a href="tags.html">Tags</a>
<div id="search-container">
<input type="text" id="search-input" placeholder="Search anywhere...">
<i class="fas fa-search search-icon"></i>
</div>
</nav>
</header></div>
<div id="content">
<div class="post-date">03 Nov 2024</div><h1 class="post-title"><a href="https://www.vandee.art/2024-11-03-readwise-and-github-actions.html">Readwise 和 GitHub Actions 联用 - 流动知识检索</a></h1>
<nav id="table-of-contents" role="doc-toc">
<h2>Table of Contents</h2>
<div id="text-table-of-contents" role="doc-toc">
<ul>
<li><a href="#org0f9255a">结合之前的 bookmark-collection</a>
<ul>
<li><a href="#org892f918">python</a></li>
<li><a href="#orgc317db5">workflow</a></li>
</ul>
</li>
<li><a href="#orgdae5ae5">Readwise highlights</a>
<ul>
<li><a href="#org2fe3de9">python</a></li>
<li><a href="#orgbd26218">workflow</a></li>
</ul>
</li>
</ul>
</div>
</nav>
<p>
Readwise 是我体验的最舒服的 after-reading 软件了，之前一直没有利用好它的 API，在 PKM 体系里，也一直没有统一到工作流。在 Claude 的协助下，有了现在的方案：
</p>
<div id="outline-container-org0f9255a" class="outline-2">
<h2 id="org0f9255a">结合之前的 bookmark-collection</h2>
<div class="outline-text-2" id="text-org0f9255a">
<p>
现在使用 <code>osmos::memos</code> 插件保存的时候，会默认也保存到 readwise。添加 <code>#2clip</code> 标签触发 clip 的 workflow。详见： <a href="https://www.vandee.art/2024-10-12-bookmark-and-summary-by-github-actions.html">用 GitHub 仓库做书签和 AI 摘要 - 流动知识检索</a>
</p>
<p>
这样就统一起来了。所有的链接都保存在 bookmark-collection 仓库作为中转站，然后统一在 Readwise 里阅读，导出 highlights。有保存价值的会触发 clip workflow 保存原文。
</p>
<p>
主要思路：在 push 的时候会运行这个 Python 脚本，读取新增内容，获取 URL 并使用 Readwise Reader 的 API 保存到 Readwise Reader。Readwise 和 Readwise Reader 是两个 API。
</p>
<p>
需要在仓库的 secret 里增加一个 <code>READWISE_TOKEN</code> secert，填入自己的 Readwise API。
</p>
</div>
<div id="outline-container-org892f918" class="outline-3">
<h3 id="org892f918">python</h3>
<div class="outline-text-3" id="text-org892f918">
<div class="org-src-container">
<pre class="src src-python"><span style="color: #FF7B72;">import</span> os
<span style="color: #FF7B72;">import</span> re
<span style="color: #FF7B72;">import</span> logging
<span style="color: #FF7B72;">import</span> requests
<span style="color: #FF7B72;">from</span> github <span style="color: #FF7B72;">import</span> Github
logging.basicConfig<span style="color: #8c8c8c;">(</span>
level=logging.INFO,
<span style="color: #79C0FF;">format</span>=<span style="color: #a7bca4;">'%(asctime)s - %(message)s'</span>,
datefmt=<span style="color: #a7bca4;">'%Y-%m-%d %H:%M:%S'</span>
<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">get_added_content</span><span style="color: #8c8c8c;">()</span>:
<span style="color: #787878; font-style: italic;">"""获取这次提交新增的内容"""</span>
<span style="color: #FF7B72;">try</span>:
<span style="color: #FFA657;">g</span> = Github<span style="color: #8c8c8c;">(</span>os.environ<span style="color: #93a8c6;">[</span><span style="color: #a7bca4;">'GITHUB_TOKEN'</span><span style="color: #93a8c6;">]</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">repo</span> = g.get_repo<span style="color: #8c8c8c;">(</span>os.environ<span style="color: #93a8c6;">[</span><span style="color: #a7bca4;">'GITHUB_REPOSITORY'</span><span style="color: #93a8c6;">]</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">commit_sha</span> = os.environ.get<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'GITHUB_SHA'</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">commit</span> = repo.get_commit<span style="color: #8c8c8c;">(</span>commit_sha<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">for</span> <span style="color: #79C0FF;">file</span> <span style="color: #FF7B72;">in</span> commit.files:
<span style="color: #FF7B72;">if</span> <span style="color: #79C0FF;">file</span>.filename.endswith<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'.md'</span><span style="color: #8c8c8c;">)</span>:
<span style="color: #FF7B72;">if</span> <span style="color: #79C0FF;">file</span>.patch <span style="color: #FF7B72;">and</span> <span style="color: #a7bca4;">'+'</span> <span style="color: #FF7B72;">in</span> <span style="color: #79C0FF;">file</span>.patch:
<span style="color: #FFA657;">added_lines</span> = <span style="color: #8c8c8c;">[</span>line<span style="color: #93a8c6;">[</span>1:<span style="color: #93a8c6;">]</span> <span style="color: #FF7B72;">for</span> line <span style="color: #FF7B72;">in</span> <span style="color: #79C0FF;">file</span>.patch.split<span style="color: #93a8c6;">(</span><span style="color: #a7bca4;">'</span><span style="font-style: italic;">\n</span><span style="color: #a7bca4;">'</span><span style="color: #93a8c6;">)</span>
<span style="color: #FF7B72;">if</span> line.startswith<span style="color: #93a8c6;">(</span><span style="color: #a7bca4;">'+'</span><span style="color: #93a8c6;">)</span> <span style="color: #FF7B72;">and</span> <span style="color: #FF7B72;">not</span> line.startswith<span style="color: #93a8c6;">(</span><span style="color: #a7bca4;">'+++'</span><span style="color: #93a8c6;">)</span><span style="color: #8c8c8c;">]</span>
<span style="color: #FF7B72;">return</span> added_lines
<span style="color: #FF7B72;">return</span> <span style="color: #8c8c8c;">[]</span>
<span style="color: #FF7B72;">except</span> <span style="color: #FF7B72; font-style: italic;">Exception</span> <span style="color: #FF7B72;">as</span> e:
logging.error<span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Error getting commit content: </span>{<span style="color: #79C0FF;">str</span>(e)}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">raise</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">extract_url</span><span style="color: #8c8c8c;">(</span>line: <span style="color: #79C0FF;">str</span><span style="color: #8c8c8c;">)</span>:
<span style="color: #787878; font-style: italic;">"""从行中提取URL，支持markdown格式的链接"""</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">更新后的URL提取模式</span>
<span style="color: #FFA657;">url_pattern</span> = r<span style="color: #a7bca4;">'\((https?://[\w\-._~:/?#\[\]@!$&\'()*+,;=.%]+)\)'</span>
<span style="color: #FF7B72;">match</span> = re.search<span style="color: #8c8c8c;">(</span>url_pattern, line<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">if</span> <span style="color: #FF7B72;">match</span>:
<span style="color: #FF7B72;">return</span> <span style="color: #FF7B72;">match</span>.group<span style="color: #8c8c8c;">(</span>1<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">return</span> <span style="font-style: italic;">None</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">has_clip_tag</span><span style="color: #8c8c8c;">(</span>line: <span style="color: #79C0FF;">str</span><span style="color: #8c8c8c;">)</span> -> <span style="color: #79C0FF;">bool</span>:
<span style="color: #787878; font-style: italic;">"""检查是否包含 #2clip 标签"""</span>
<span style="color: #FF7B72;">return</span> <span style="color: #79C0FF;">bool</span><span style="color: #8c8c8c;">(</span>re.search<span style="color: #93a8c6;">(</span>r<span style="color: #a7bca4;">'#2clip\b'</span>, line<span style="color: #93a8c6;">)</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">trigger_workflow</span><span style="color: #8c8c8c;">()</span>:
<span style="color: #787878; font-style: italic;">"""触发另一个workflow"""</span>
<span style="color: #FF7B72;">try</span>:
<span style="color: #FFA657;">g</span> = Github<span style="color: #8c8c8c;">(</span>os.environ<span style="color: #93a8c6;">[</span><span style="color: #a7bca4;">'GITHUB_TOKEN'</span><span style="color: #93a8c6;">]</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">repo</span> = g.get_repo<span style="color: #8c8c8c;">(</span>os.environ<span style="color: #93a8c6;">[</span><span style="color: #a7bca4;">'GITHUB_REPOSITORY'</span><span style="color: #93a8c6;">]</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">workflow</span> = repo.get_workflow<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"bookmark_summary.yml"</span><span style="color: #8c8c8c;">)</span>
workflow.create_dispatch<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"main"</span><span style="color: #8c8c8c;">)</span>
logging.info<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"Successfully triggered the bookmark_summary workflow"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">except</span> <span style="color: #FF7B72; font-style: italic;">Exception</span> <span style="color: #FF7B72;">as</span> e:
logging.error<span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Failed to trigger workflow: </span>{<span style="color: #79C0FF;">str</span>(e)}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">raise</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">main</span><span style="color: #8c8c8c;">()</span>:
<span style="color: #FF7B72;">try</span>:
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">获取所有新增的内容</span>
<span style="color: #FFA657;">added_lines</span> = get_added_content<span style="color: #8c8c8c;">()</span>
<span style="color: #FF7B72;">if</span> <span style="color: #FF7B72;">not</span> added_lines:
logging.info<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"No new markdown content found in this commit"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">return</span>
<span style="color: #FFA657;">trigger_needed</span> = <span style="font-style: italic;">False</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">处理每一行新增的内容</span>
<span style="color: #FF7B72;">for</span> line <span style="color: #FF7B72;">in</span> added_lines:
<span style="color: #FFA657;">line</span> = line.strip<span style="color: #8c8c8c;">()</span>
logging.info<span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Processing line: </span>{line}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">检查标签</span>
<span style="color: #FF7B72;">if</span> has_clip_tag<span style="color: #8c8c8c;">(</span>line<span style="color: #8c8c8c;">)</span>:
<span style="color: #FFA657;">trigger_needed</span> = <span style="font-style: italic;">True</span>
logging.info<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"Found #2clip tag"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">提取并处理URL（无论是否有标签）</span>
<span style="color: #FFA657;">url</span> = extract_url<span style="color: #8c8c8c;">(</span>line<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">if</span> url:
<span style="color: #FF7B72;">try</span>:
<span style="color: #FFA657;">response</span> = requests.post<span style="color: #8c8c8c;">(</span>
url=<span style="color: #a7bca4;">"https://readwise.io/api/v3/save/"</span>,
headers=<span style="color: #93a8c6;">{</span><span style="color: #a7bca4;">"Authorization"</span>: f<span style="color: #a7bca4;">"Token </span>{os.environ['READWISE_TOKEN']}<span style="color: #a7bca4;">"</span><span style="color: #93a8c6;">}</span>,
json=<span style="color: #93a8c6;">{</span>
<span style="color: #a7bca4;">"url"</span>: url,
<span style="color: #a7bca4;">"tags"</span>: <span style="color: #b0b1a3;">[</span><span style="color: #a7bca4;">"Bookmark"</span><span style="color: #b0b1a3;">]</span>
<span style="color: #93a8c6;">}</span>
<span style="color: #8c8c8c;">)</span>
response.raise_for_status<span style="color: #8c8c8c;">()</span>
logging.info<span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Successfully saved URL: </span>{url}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">except</span> requests.exceptions.RequestException <span style="color: #FF7B72;">as</span> e:
logging.error<span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Failed to save URL </span>{url}<span style="color: #a7bca4;">: </span>{<span style="color: #79C0FF;">str</span>(e)}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">如果发现了标签，触发workflow</span>
<span style="color: #FF7B72;">if</span> trigger_needed:
logging.info<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"Triggering workflow due to #2clip tag"</span><span style="color: #8c8c8c;">)</span>
trigger_workflow<span style="color: #8c8c8c;">()</span>
<span style="color: #FF7B72;">except</span> <span style="color: #FF7B72; font-style: italic;">Exception</span> <span style="color: #FF7B72;">as</span> e:
logging.error<span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Error: </span>{<span style="color: #79C0FF;">str</span>(e)}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">raise</span>
<span style="color: #FF7B72;">if</span> <span style="color: #79C0FF;">__name__</span> == <span style="color: #a7bca4;">"__main__"</span>:
main<span style="color: #8c8c8c;">()</span>
</pre>
</div>
</div>
</div>
<div id="outline-container-orgc317db5" class="outline-3">
<h3 id="orgc317db5">workflow</h3>
<div class="outline-text-3" id="text-orgc317db5">
<div class="org-src-container">
<pre class="src src-yaml">
<span style="color: #FFA657;">name</span>: Save Bookmark to Readwise
<span style="font-style: italic;">on</span>:
<span style="color: #FFA657;">push</span>:
<span style="color: #FFA657;">branches</span>:
- main
<span style="color: #FFA657;">paths</span>:
- <span style="color: #a7bca4;">'**.md'</span>
<span style="color: #FFA657;">workflow_dispatch</span>:
<span style="color: #FFA657;">permissions</span>:
<span style="color: #FFA657;">contents</span>: read
<span style="color: #FFA657;">actions</span>: write
<span style="color: #FFA657;">jobs</span>:
<span style="color: #FFA657;">save-to-readwise</span>:
<span style="color: #FFA657;">runs-on</span>: ubuntu-latest
<span style="color: #FFA657;">steps</span>:
- <span style="color: #FFA657;">name</span>: Checkout repository
<span style="color: #FFA657;">uses</span>: actions/checkout@v4
<span style="color: #FFA657;">with</span>:
<span style="color: #FFA657;">token</span>: ${{ secrets.GITHUB_TOKEN }}
- <span style="color: #FFA657;">name</span>: Set up Python
<span style="color: #FFA657;">uses</span>: actions/setup-python@v4
<span style="color: #FFA657;">with</span>:
<span style="color: #FFA657;">python-version</span>: <span style="color: #a7bca4;">'3.10'</span>
- <span style="color: #FFA657;">name</span>: Install dependencies
<span style="color: #FFA657;">run</span>: |
<span style="color: #a7bca4;">python -m pip install --upgrade pip</span>
<span style="color: #a7bca4;"> pip install requests PyGithub</span>
- <span style="color: #FFA657;">name</span>: Run bookmark saver
<span style="color: #FFA657;">env</span>:
<span style="color: #FFA657;">READWISE_TOKEN</span>: ${{ secrets.READWISE_TOKEN }}
<span style="color: #FFA657;">GITHUB_TOKEN</span>: ${{ secrets.GITHUB_TOKEN }}
<span style="color: #FFA657;">GITHUB_REPOSITORY</span>: ${{ github.repository }}
<span style="color: #FFA657;">run</span>: python save_to_readwise.py
</pre>
</div>
</div>
</div>
</div>
<div id="outline-container-orgdae5ae5" class="outline-2">
<h2 id="orgdae5ae5">Readwise highlights</h2>
<div class="outline-text-2" id="text-orgdae5ae5">
<p>
写了一个 <code>class ReadwiseAPI</code> 方便其他项目引入。可以定时获取我所有 highlights 的 title 和 url。
</p>
</div>
<div id="outline-container-org2fe3de9" class="outline-3">
<h3 id="org2fe3de9">python</h3>
<div class="outline-text-3" id="text-org2fe3de9">
<div class="org-src-container">
<pre class="src src-python">
<span style="color: #FF7B72;">import</span> requests
<span style="color: #FF7B72;">import</span> json
<span style="color: #FF7B72;">from</span> datetime <span style="color: #FF7B72;">import</span> datetime, timedelta
<span style="color: #FF7B72;">import</span> os
<span style="color: #FF7B72;">from</span> typing <span style="color: #FF7B72;">import</span> List, Dict, Optional
<span style="color: #FF7B72;">from</span> pathlib <span style="color: #FF7B72;">import</span> Path
<span style="color: #FF7B72;">import</span> re
<span style="color: #FF7B72;">from</span> github <span style="color: #FF7B72;">import</span> Github
<span style="color: #FF7B72;">import</span> argparse
<span style="color: #FF7B72;">class</span> <span style="color: #FF7B72; font-style: italic;">ReadwiseAPI</span>:
<span style="color: #787878; font-style: italic;">"""Readwise API client for exporting highlights with smart update capability and GitHub integration"""</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">__init__</span><span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span><span style="color: #8c8c8c;">)</span>:
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Initialize Readwise token</span>
<span style="color: #FF7B72;">self</span>.<span style="color: #FFA657;">readwise_token</span> = os.environ.get<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"READWISE_TOKEN"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">if</span> <span style="color: #FF7B72;">not</span> <span style="color: #FF7B72;">self</span>.readwise_token:
<span style="color: #FF7B72;">raise</span> <span style="color: #FF7B72; font-style: italic;">ValueError</span><span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"READWISE_TOKEN not found in environment variables"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Initialize GitHub token</span>
<span style="color: #FF7B72;">self</span>.<span style="color: #FFA657;">github_token</span> = os.environ.get<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"GITHUB_TOKEN"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">if</span> <span style="color: #FF7B72;">not</span> <span style="color: #FF7B72;">self</span>.github_token:
<span style="color: #FF7B72;">raise</span> <span style="color: #FF7B72; font-style: italic;">ValueError</span><span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"GITHUB_TOKEN not found in environment variables"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Get repository from GitHub Actions environment variable</span>
<span style="color: #FF7B72;">self</span>.<span style="color: #FFA657;">github_repo</span> = os.environ.get<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"GITHUB_REPOSITORY"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">if</span> <span style="color: #FF7B72;">not</span> <span style="color: #FF7B72;">self</span>.github_repo:
<span style="color: #FF7B72;">raise</span> <span style="color: #FF7B72; font-style: italic;">ValueError</span><span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"Not running in GitHub Actions environment (GITHUB_REPOSITORY not found)"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Initialize GitHub client</span>
<span style="color: #FF7B72;">self</span>.<span style="color: #FFA657;">github</span> = Github<span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span>.github_token<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">self</span>.<span style="color: #FFA657;">repo</span> = <span style="color: #FF7B72;">self</span>.github.get_repo<span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span>.github_repo<span style="color: #8c8c8c;">)</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Initialize Readwise API settings</span>
<span style="color: #FF7B72;">self</span>.<span style="color: #FFA657;">base_url</span> = <span style="color: #a7bca4;">"https://readwise.io/api/v2"</span>
<span style="color: #FF7B72;">self</span>.<span style="color: #FFA657;">headers</span> = <span style="color: #8c8c8c;">{</span>
<span style="color: #a7bca4;">"Authorization"</span>: f<span style="color: #a7bca4;">"Token </span>{<span style="color: #FF7B72;">self</span>.readwise_token}<span style="color: #a7bca4;">"</span>
<span style="color: #8c8c8c;">}</span>
<span style="color: #FF7B72;">self</span>.<span style="color: #FFA657;">last_update_file</span> = <span style="color: #a7bca4;">"last_update.json"</span>
<span style="color: #FF7B72;">self</span>.<span style="color: #FFA657;">articles_file</span> = <span style="color: #a7bca4;">"articles.json"</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">get_highlights</span><span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span>, updated_after: Optional<span style="color: #93a8c6;">[</span>datetime<span style="color: #93a8c6;">]</span> = <span style="font-style: italic;">None</span>,
start_date: Optional<span style="color: #93a8c6;">[</span>datetime<span style="color: #93a8c6;">]</span> = <span style="font-style: italic;">None</span>,
end_date: Optional<span style="color: #93a8c6;">[</span>datetime<span style="color: #93a8c6;">]</span> = <span style="font-style: italic;">None</span><span style="color: #8c8c8c;">)</span> -> Dict:
<span style="color: #787878; font-style: italic;">"""Get all highlights with their associated metadata"""</span>
<span style="color: #FFA657;">endpoint</span> = f<span style="color: #a7bca4;">"</span>{<span style="color: #FF7B72;">self</span>.base_url}<span style="color: #a7bca4;">/export/"</span>
<span style="color: #FFA657;">params</span> = <span style="color: #8c8c8c;">{}</span>
<span style="color: #FF7B72;">if</span> updated_after:
<span style="color: #FFA657;">params</span><span style="color: #8c8c8c;">[</span><span style="color: #a7bca4;">"updated_after"</span><span style="color: #8c8c8c;">]</span> = updated_after.isoformat<span style="color: #8c8c8c;">()</span>
<span style="color: #FF7B72;">elif</span> start_date:
<span style="color: #FFA657;">params</span><span style="color: #8c8c8c;">[</span><span style="color: #a7bca4;">"updated_after"</span><span style="color: #8c8c8c;">]</span> = start_date.isoformat<span style="color: #8c8c8c;">()</span>
<span style="color: #FF7B72;">if</span> end_date:
<span style="color: #FFA657;">params</span><span style="color: #8c8c8c;">[</span><span style="color: #a7bca4;">"updated_before"</span><span style="color: #8c8c8c;">]</span> = end_date.isoformat<span style="color: #8c8c8c;">()</span>
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Fetching highlights with params: </span>{params}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">response</span> = requests.get<span style="color: #8c8c8c;">(</span>endpoint, headers=<span style="color: #FF7B72;">self</span>.headers, params=params<span style="color: #8c8c8c;">)</span>
response.raise_for_status<span style="color: #8c8c8c;">()</span>
<span style="color: #FF7B72;">return</span> response.json<span style="color: #8c8c8c;">()</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">get_file_content</span><span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span>, path: <span style="color: #79C0FF;">str</span><span style="color: #8c8c8c;">)</span> -> Optional<span style="color: #8c8c8c;">[</span><span style="color: #79C0FF;">str</span><span style="color: #8c8c8c;">]</span>:
<span style="color: #787878; font-style: italic;">"""Get file content from GitHub repository"""</span>
<span style="color: #FF7B72;">try</span>:
<span style="color: #FFA657;">content</span> = <span style="color: #FF7B72;">self</span>.repo.get_contents<span style="color: #8c8c8c;">(</span>path<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">return</span> content.decoded_content.decode<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'utf-8'</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">except</span> <span style="color: #FF7B72; font-style: italic;">Exception</span> <span style="color: #FF7B72;">as</span> e:
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"File </span>{path}<span style="color: #a7bca4;"> not found in repository: </span>{e}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">return</span> <span style="font-style: italic;">None</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">update_file</span><span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span>, path: <span style="color: #79C0FF;">str</span>, content: <span style="color: #79C0FF;">str</span>, message: <span style="color: #79C0FF;">str</span><span style="color: #8c8c8c;">)</span>:
<span style="color: #787878; font-style: italic;">"""Update or create file in GitHub repository"""</span>
<span style="color: #FF7B72;">try</span>:
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Try to get existing file</span>
<span style="color: #79C0FF;">file</span> = <span style="color: #FF7B72;">self</span>.repo.get_contents<span style="color: #8c8c8c;">(</span>path<span style="color: #8c8c8c;">)</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Update existing file</span>
<span style="color: #FF7B72;">self</span>.repo.update_file<span style="color: #8c8c8c;">(</span>
path=path,
message=message,
content=content,
sha=<span style="color: #79C0FF;">file</span>.sha
<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">except</span> <span style="color: #FF7B72; font-style: italic;">Exception</span>:
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Create new file if it doesn't exist</span>
<span style="color: #FF7B72;">self</span>.repo.create_file<span style="color: #8c8c8c;">(</span>
path=path,
message=message,
content=content
<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">clean_title</span><span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span>, title: <span style="color: #79C0FF;">str</span><span style="color: #8c8c8c;">)</span> -> <span style="color: #79C0FF;">str</span>:
<span style="color: #787878; font-style: italic;">"""Clean title by removing newlines and extra spaces"""</span>
<span style="color: #FFA657;">title</span> = re.sub<span style="color: #8c8c8c;">(</span>r<span style="color: #a7bca4;">'\s+'</span>, <span style="color: #a7bca4;">' '</span>, title.replace<span style="color: #93a8c6;">(</span><span style="color: #a7bca4;">'</span><span style="font-style: italic;">\n</span><span style="color: #a7bca4;">'</span>, <span style="color: #a7bca4;">' '</span><span style="color: #93a8c6;">)</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">return</span> title.strip<span style="color: #8c8c8c;">()</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">create_article_json</span><span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span>, highlights_data: Dict<span style="color: #8c8c8c;">)</span> -> List<span style="color: #8c8c8c;">[</span>Dict<span style="color: #8c8c8c;">]</span>:
<span style="color: #787878; font-style: italic;">"""Create a list of articles with title and URL, only for category 'articles'"""</span>
<span style="color: #FFA657;">articles</span> = <span style="color: #8c8c8c;">[]</span>
<span style="color: #FF7B72;">for</span> article <span style="color: #FF7B72;">in</span> highlights_data.get<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'results'</span>, <span style="color: #93a8c6;">[]</span><span style="color: #8c8c8c;">)</span>:
<span style="color: #FF7B72;">if</span> article.get<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'category'</span>, <span style="color: #a7bca4;">''</span><span style="color: #8c8c8c;">)</span>.lower<span style="color: #8c8c8c;">()</span> == <span style="color: #a7bca4;">'articles'</span>:
<span style="color: #FFA657;">title</span> = <span style="color: #FF7B72;">self</span>.clean_title<span style="color: #8c8c8c;">(</span>article.get<span style="color: #93a8c6;">(</span><span style="color: #a7bca4;">'title'</span>, <span style="color: #a7bca4;">'Untitled'</span><span style="color: #93a8c6;">)</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">url</span> = article.get<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'source_url'</span>, <span style="color: #a7bca4;">''</span><span style="color: #8c8c8c;">)</span>
articles.append<span style="color: #8c8c8c;">(</span><span style="color: #93a8c6;">{</span>
<span style="color: #a7bca4;">'title'</span>: title,
<span style="color: #a7bca4;">'url'</span>: url
<span style="color: #93a8c6;">}</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">return</span> articles
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">load_last_update_from_github</span><span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span><span style="color: #8c8c8c;">)</span> -> Optional<span style="color: #8c8c8c;">[</span>datetime<span style="color: #8c8c8c;">]</span>:
<span style="color: #787878; font-style: italic;">"""Load the last update date from GitHub"""</span>
<span style="color: #FFA657;">content</span> = <span style="color: #FF7B72;">self</span>.get_file_content<span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span>.last_update_file<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">if</span> content:
<span style="color: #FF7B72;">try</span>:
<span style="color: #FFA657;">data</span> = json.loads<span style="color: #8c8c8c;">(</span>content<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">return</span> datetime.strptime<span style="color: #8c8c8c;">(</span>data<span style="color: #93a8c6;">[</span><span style="color: #a7bca4;">'last_update'</span><span style="color: #93a8c6;">]</span>, <span style="color: #a7bca4;">'%Y-%m-%d'</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">except</span> <span style="color: #FF7B72; font-style: italic;">Exception</span> <span style="color: #FF7B72;">as</span> e:
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Error parsing last update file: </span>{e}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">return</span> <span style="font-style: italic;">None</span>
<span style="color: #FF7B72;">return</span> <span style="font-style: italic;">None</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">save_last_update_to_github</span><span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span><span style="color: #8c8c8c;">)</span>:
<span style="color: #787878; font-style: italic;">"""Save current date as last update date to GitHub"""</span>
<span style="color: #FFA657;">current_date</span> = datetime.now<span style="color: #8c8c8c;">()</span>.strftime<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'%Y-%m-%d'</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">content</span> = json.dumps<span style="color: #8c8c8c;">(</span><span style="color: #93a8c6;">{</span><span style="color: #a7bca4;">'last_update'</span>: current_date<span style="color: #93a8c6;">}</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">self</span>.update_file<span style="color: #8c8c8c;">(</span>
path=<span style="color: #FF7B72;">self</span>.last_update_file,
content=content,
message=<span style="color: #a7bca4;">"Update last sync date"</span>
<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">load_existing_articles_from_github</span><span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span><span style="color: #8c8c8c;">)</span> -> List<span style="color: #8c8c8c;">[</span>Dict<span style="color: #8c8c8c;">]</span>:
<span style="color: #787878; font-style: italic;">"""Load existing articles from GitHub"""</span>
<span style="color: #FFA657;">content</span> = <span style="color: #FF7B72;">self</span>.get_file_content<span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span>.articles_file<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">if</span> content:
<span style="color: #FF7B72;">try</span>:
<span style="color: #FF7B72;">return</span> json.loads<span style="color: #8c8c8c;">(</span>content<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">except</span> <span style="color: #FF7B72; font-style: italic;">Exception</span> <span style="color: #FF7B72;">as</span> e:
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Error parsing articles file: </span>{e}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">return</span> <span style="color: #8c8c8c;">[]</span>
<span style="color: #FF7B72;">return</span> <span style="color: #8c8c8c;">[]</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">merge_articles</span><span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span>, existing_articles: List<span style="color: #93a8c6;">[</span>Dict<span style="color: #93a8c6;">]</span>, new_articles: List<span style="color: #93a8c6;">[</span>Dict<span style="color: #93a8c6;">]</span><span style="color: #8c8c8c;">)</span> -> List<span style="color: #8c8c8c;">[</span>Dict<span style="color: #8c8c8c;">]</span>:
<span style="color: #787878; font-style: italic;">"""Merge new articles with existing ones, avoiding duplicates"""</span>
<span style="color: #FFA657;">existing_set</span> = <span style="color: #8c8c8c;">{</span><span style="color: #93a8c6;">(</span>article<span style="color: #b0b1a3;">[</span><span style="color: #a7bca4;">'title'</span><span style="color: #b0b1a3;">]</span>, article<span style="color: #b0b1a3;">[</span><span style="color: #a7bca4;">'url'</span><span style="color: #b0b1a3;">]</span><span style="color: #93a8c6;">)</span> <span style="color: #FF7B72;">for</span> article <span style="color: #FF7B72;">in</span> existing_articles<span style="color: #8c8c8c;">}</span>
<span style="color: #FF7B72;">for</span> article <span style="color: #FF7B72;">in</span> new_articles:
<span style="color: #FFA657;">article_tuple</span> = <span style="color: #8c8c8c;">(</span>article<span style="color: #93a8c6;">[</span><span style="color: #a7bca4;">'title'</span><span style="color: #93a8c6;">]</span>, article<span style="color: #93a8c6;">[</span><span style="color: #a7bca4;">'url'</span><span style="color: #93a8c6;">]</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">if</span> article_tuple <span style="color: #FF7B72;">not</span> <span style="color: #FF7B72;">in</span> existing_set:
existing_articles.append<span style="color: #8c8c8c;">(</span>article<span style="color: #8c8c8c;">)</span>
existing_set.add<span style="color: #8c8c8c;">(</span>article_tuple<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">return</span> existing_articles
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">export_articles</span><span style="color: #8c8c8c;">(</span><span style="color: #FF7B72;">self</span>, start_date: Optional<span style="color: #93a8c6;">[</span><span style="color: #79C0FF;">str</span><span style="color: #93a8c6;">]</span> = <span style="font-style: italic;">None</span>,
end_date: Optional<span style="color: #93a8c6;">[</span><span style="color: #79C0FF;">str</span><span style="color: #93a8c6;">]</span> = <span style="font-style: italic;">None</span>,
all_time: <span style="color: #79C0FF;">bool</span> = <span style="font-style: italic;">False</span><span style="color: #8c8c8c;">)</span>:
<span style="color: #787878; font-style: italic;">"""</span>
<span style="color: #787878; font-style: italic;"> Export articles to GitHub with smart update capability</span>
<span style="color: #787878; font-style: italic;"> Args:</span>
<span style="color: #787878; font-style: italic;"> start_date: Optional start date in YYYY-MM-DD format</span>
<span style="color: #787878; font-style: italic;"> end_date: Optional end date in YYYY-MM-DD format</span>
<span style="color: #787878; font-style: italic;"> all_time: If True, fetch all highlights regardless of dates</span>
<span style="color: #787878; font-style: italic;"> """</span>
<span style="color: #FF7B72;">if</span> all_time:
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">当选择 all_time 时，强制获取所有 highlights，忽略上次更新时间</span>
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"Fetching all highlights from the beginning"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">highlights_data</span> = <span style="color: #FF7B72;">self</span>.get_highlights<span style="color: #8c8c8c;">()</span>
<span style="color: #FF7B72;">elif</span> start_date:
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">如果指定了开始日期，使用指定的日期范围</span>
<span style="color: #FFA657;">start_datetime</span> = datetime.strptime<span style="color: #8c8c8c;">(</span>start_date, <span style="color: #a7bca4;">'%Y-%m-%d'</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">end_datetime</span> = datetime.strptime<span style="color: #8c8c8c;">(</span>end_date, <span style="color: #a7bca4;">'%Y-%m-%d'</span><span style="color: #8c8c8c;">)</span> <span style="color: #FF7B72;">if</span> end_date <span style="color: #FF7B72;">else</span> datetime.now<span style="color: #8c8c8c;">()</span>
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Fetching highlights from </span>{start_date}<span style="color: #a7bca4;"> to </span>{end_date <span style="color: #FF7B72;">or</span> 'now'}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">highlights_data</span> = <span style="color: #FF7B72;">self</span>.get_highlights<span style="color: #8c8c8c;">(</span>start_date=start_datetime, end_date=end_datetime<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">else</span>:
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">使用上次更新时间的增量更新逻辑</span>
<span style="color: #FFA657;">last_update</span> = <span style="color: #FF7B72;">self</span>.load_last_update_from_github<span style="color: #8c8c8c;">()</span>
<span style="color: #FF7B72;">if</span> last_update:
<span style="color: #FFA657;">days_since_update</span> = <span style="color: #8c8c8c;">(</span>datetime.now<span style="color: #93a8c6;">()</span> - last_update<span style="color: #8c8c8c;">)</span>.days
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Last update was </span>{days_since_update}<span style="color: #a7bca4;"> days ago on </span>{last_update.strftime('%Y-%m-%d')}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">if</span> days_since_update > 0:
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Fetching highlights updated after </span>{last_update.strftime('%Y-%m-%d')}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">highlights_data</span> = <span style="color: #FF7B72;">self</span>.get_highlights<span style="color: #8c8c8c;">(</span>updated_after=last_update<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">else</span>:
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"Already updated today, no need to fetch new articles"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">return</span>
<span style="color: #FF7B72;">else</span>:
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"No previous update found, fetching all articles"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">highlights_data</span> = <span style="color: #FF7B72;">self</span>.get_highlights<span style="color: #8c8c8c;">()</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Create article data</span>
<span style="color: #FFA657;">new_articles</span> = <span style="color: #FF7B72;">self</span>.create_article_json<span style="color: #8c8c8c;">(</span>highlights_data<span style="color: #8c8c8c;">)</span>
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Found </span>{<span style="color: #79C0FF;">len</span>(new_articles)}<span style="color: #a7bca4;"> new articles"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Load existing articles</span>
<span style="color: #FFA657;">existing_articles</span> = <span style="color: #FF7B72;">self</span>.load_existing_articles_from_github<span style="color: #8c8c8c;">()</span>
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Found </span>{<span style="color: #79C0FF;">len</span>(existing_articles)}<span style="color: #a7bca4;"> existing articles"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Merge new articles with existing ones</span>
<span style="color: #FFA657;">merged_articles</span> = <span style="color: #FF7B72;">self</span>.merge_articles<span style="color: #8c8c8c;">(</span>existing_articles, new_articles<span style="color: #8c8c8c;">)</span>
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Total unique articles after merge: </span>{<span style="color: #79C0FF;">len</span>(merged_articles)}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Save merged articles to GitHub</span>
<span style="color: #FF7B72;">self</span>.update_file<span style="color: #8c8c8c;">(</span>
path=<span style="color: #FF7B72;">self</span>.articles_file,
content=json.dumps<span style="color: #93a8c6;">(</span>merged_articles, ensure_ascii=<span style="font-style: italic;">False</span>, indent=2<span style="color: #93a8c6;">)</span>,
message=<span style="color: #a7bca4;">"Update articles list"</span>
<span style="color: #8c8c8c;">)</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">Update the last update date</span>
<span style="color: #FF7B72;">if</span> <span style="color: #FF7B72;">not</span> start_date <span style="color: #FF7B72;">and</span> <span style="color: #FF7B72;">not</span> all_time: <span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">只有在非指定日期范围和非全量更新的情况下才更新最后同步时间</span>
<span style="color: #FF7B72;">self</span>.save_last_update_to_github<span style="color: #8c8c8c;">()</span>
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"Successfully updated articles in GitHub repository"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">if</span> new_articles:
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">"New articles added:"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">for</span> article <span style="color: #FF7B72;">in</span> new_articles:
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"- </span>{article['title']}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">def</span> <span style="color: #D2A8FF;">main</span><span style="color: #8c8c8c;">()</span>:
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">从环境变量获取 GitHub Actions 的输入参数</span>
<span style="color: #FFA657;">gh_start_date</span> = os.environ.get<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'INPUT_START_DATE'</span>, <span style="color: #a7bca4;">''</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">gh_end_date</span> = os.environ.get<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'INPUT_END_DATE'</span>, <span style="color: #a7bca4;">''</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">gh_all_time</span> = os.environ.get<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'INPUT_ALL_TIME'</span>, <span style="color: #a7bca4;">''</span><span style="color: #8c8c8c;">)</span>.lower<span style="color: #8c8c8c;">()</span> == <span style="color: #a7bca4;">'true'</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">设置命令行参数解析器</span>
<span style="color: #FFA657;">parser</span> = argparse.ArgumentParser<span style="color: #8c8c8c;">(</span>description=<span style="color: #a7bca4;">'Sync Readwise highlights to GitHub'</span><span style="color: #8c8c8c;">)</span>
parser.add_argument<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'--start-date'</span>, <span style="color: #79C0FF;">type</span>=<span style="color: #79C0FF;">str</span>, <span style="color: #79C0FF;">help</span>=<span style="color: #a7bca4;">'Start date in YYYY-MM-DD format'</span><span style="color: #8c8c8c;">)</span>
parser.add_argument<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'--end-date'</span>, <span style="color: #79C0FF;">type</span>=<span style="color: #79C0FF;">str</span>, <span style="color: #79C0FF;">help</span>=<span style="color: #a7bca4;">'End date in YYYY-MM-DD format'</span><span style="color: #8c8c8c;">)</span>
parser.add_argument<span style="color: #8c8c8c;">(</span><span style="color: #a7bca4;">'--all-time'</span>, action=<span style="color: #a7bca4;">'store_true'</span>, <span style="color: #79C0FF;">help</span>=<span style="color: #a7bca4;">'Fetch all highlights from the beginning'</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FFA657;">args</span> = parser.parse_args<span style="color: #8c8c8c;">()</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">优先使用命令行参数，如果没有则使用 GitHub Actions 的输入参数</span>
<span style="color: #FFA657;">start_date</span> = args.start_date <span style="color: #FF7B72;">or</span> gh_start_date
<span style="color: #FFA657;">end_date</span> = args.end_date <span style="color: #FF7B72;">or</span> gh_end_date
<span style="color: #FFA657;">all_time</span> = args.all_time <span style="color: #FF7B72;">or</span> gh_all_time
<span style="color: #FF7B72;">try</span>:
<span style="color: #FFA657;">client</span> = ReadwiseAPI<span style="color: #8c8c8c;">()</span>
client.export_articles<span style="color: #8c8c8c;">(</span>
start_date=start_date <span style="color: #FF7B72;">if</span> start_date <span style="color: #FF7B72;">else</span> <span style="font-style: italic;">None</span>,
end_date=end_date <span style="color: #FF7B72;">if</span> end_date <span style="color: #FF7B72;">else</span> <span style="font-style: italic;">None</span>,
all_time=all_time
<span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">except</span> <span style="color: #FF7B72; font-style: italic;">Exception</span> <span style="color: #FF7B72;">as</span> e:
<span style="color: #79C0FF;">print</span><span style="color: #8c8c8c;">(</span>f<span style="color: #a7bca4;">"An error occurred: </span>{<span style="color: #79C0FF;">str</span>(e)}<span style="color: #a7bca4;">"</span><span style="color: #8c8c8c;">)</span>
<span style="color: #FF7B72;">raise</span>
<span style="color: #FF7B72;">if</span> <span style="color: #79C0FF;">__name__</span> == <span style="color: #a7bca4;">"__main__"</span>:
main<span style="color: #8c8c8c;">()</span>
</pre>
</div>
</div>
</div>
<div id="outline-container-orgbd26218" class="outline-3">
<h3 id="orgbd26218">workflow</h3>
<div class="outline-text-3" id="text-orgbd26218">
<div class="org-src-container">
<pre class="src src-yaml">
<span style="color: #FFA657;">name</span>: Sync Readwise Articles
<span style="font-style: italic;">on</span>:
<span style="color: #FFA657;">schedule</span>:
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">每天凌晨 1 点运行 (UTC 时间，对应北京时间 9 点)</span>
- <span style="color: #FFA657;">cron</span>: <span style="color: #a7bca4;">'0 1 * * *'</span>
<span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">支持手动触发，并添加输入参数</span>
<span style="color: #FFA657;">workflow_dispatch</span>:
<span style="color: #FFA657;">inputs</span>:
<span style="color: #FFA657;">start_date</span>:
<span style="color: #FFA657;">description</span>: <span style="color: #a7bca4;">'Start date (YYYY-MM-DD, e.g., 2024-01-01)'</span>
<span style="color: #FFA657;">required</span>: <span style="font-style: italic;">false</span>
<span style="color: #FFA657;">type</span>: string
<span style="color: #FFA657;">default</span>: <span style="color: #a7bca4;">''</span>
<span style="color: #FFA657;">end_date</span>:
<span style="color: #FFA657;">description</span>: <span style="color: #a7bca4;">'End date (YYYY-MM-DD, leave empty for current date)'</span>
<span style="color: #FFA657;">required</span>: <span style="font-style: italic;">false</span>
<span style="color: #FFA657;">type</span>: string
<span style="color: #FFA657;">default</span>: <span style="color: #a7bca4;">''</span>
<span style="color: #FFA657;">all_time</span>:
<span style="color: #FFA657;">description</span>: <span style="color: #a7bca4;">'Fetch all highlights (overrides date range if selected)'</span>
<span style="color: #FFA657;">type</span>: boolean
<span style="color: #FFA657;">required</span>: <span style="font-style: italic;">false</span>
<span style="color: #FFA657;">default</span>: <span style="font-style: italic;">false</span>
<span style="color: #FFA657;">permissions</span>:
<span style="color: #FFA657;">contents</span>: write <span style="color: #787878; font-style: italic;"># </span><span style="color: #787878; font-style: italic;">仓库内容的读写权限</span>
<span style="color: #FFA657;">jobs</span>:
<span style="color: #FFA657;">sync</span>:
<span style="color: #FFA657;">runs-on</span>: ubuntu-latest
<span style="color: #FFA657;">steps</span>:
- <span style="color: #FFA657;">name</span>: Checkout repository
<span style="color: #FFA657;">uses</span>: actions/checkout@v4
- <span style="color: #FFA657;">name</span>: Set up Python
<span style="color: #FFA657;">uses</span>: actions/setup-python@v5
<span style="color: #FFA657;">with</span>:
<span style="color: #FFA657;">python-version</span>: <span style="color: #a7bca4;">'3.10'</span>
<span style="color: #FFA657;">cache</span>: <span style="color: #a7bca4;">'pip'</span>
<span style="color: #FFA657;">cache-dependency-path</span>: <span style="color: #a7bca4;">'**/requirements.txt'</span>
- <span style="color: #FFA657;">name</span>: Install dependencies
<span style="color: #FFA657;">run</span>: |
<span style="color: #a7bca4;">python -m pip install --upgrade pip</span>
<span style="color: #a7bca4;"> pip install -r requirements.txt</span>
- <span style="color: #FFA657;">name</span>: Run sync script
<span style="color: #FFA657;">env</span>:
<span style="color: #FFA657;">READWISE_TOKEN</span>: ${{ secrets.READWISE_TOKEN }}
<span style="color: #FFA657;">GITHUB_TOKEN</span>: ${{ secrets.GITHUB_TOKEN }}
<span style="color: #FFA657;">INPUT_START_DATE</span>: ${{ github.event.inputs.start_date }}
<span style="color: #FFA657;">INPUT_END_DATE</span>: ${{ github.event.inputs.end_date }}
<span style="color: #FFA657;">INPUT_ALL_TIME</span>: ${{ github.event.inputs.all_time }}
<span style="color: #FFA657;">run</span>: python readwise_sync.py
- <span style="color: #FFA657;">name</span>: Check for changes
<span style="color: #FFA657;">id</span>: verify-changed-files
<span style="color: #FFA657;">run</span>: |
if [ -n <span style="color: #a7bca4;">"$(git status --porcelain)"</span> ]; then
echo <span style="color: #a7bca4;">"changes_found=true"</span> >> $GITHUB_OUTPUT
<span style="color: #a7bca4;"> else</span>
echo <span style="color: #a7bca4;">"changes_found=false"</span> >> $GITHUB_OUTPUT
<span style="color: #a7bca4;"> fi</span>
- <span style="color: #FFA657;">name</span>: Commit changes
<span style="color: #FFA657;">if</span>: steps.verify-changed-files.outputs.changes_found == <span style="color: #a7bca4;">'true'</span>
<span style="color: #FFA657;">run</span>: |
git config --local user.email <span style="color: #a7bca4;">"github-actions[bot]@users.noreply.github.com"</span>
git config --local user.name <span style="color: #a7bca4;">"github-actions[bot]"</span>
<span style="color: #a7bca4;"> git add articles.json last_update.json</span>
git commit -m <span style="color: #a7bca4;">"Update Readwise articles [skip ci]"</span> || echo <span style="color: #a7bca4;">"No changes to commit"</span>
- <span style="color: #FFA657;">name</span>: Push changes
<span style="color: #FFA657;">if</span>: steps.verify-changed-files.outputs.changes_found == <span style="color: #a7bca4;">'true'</span>
<span style="color: #FFA657;">uses</span>: ad-m/github-push-action@master
<span style="color: #FFA657;">with</span>:
<span style="color: #FFA657;">github_token</span>: ${{ secrets.GITHUB_TOKEN }}
<span style="color: #FFA657;">branch</span>: ${{ github.ref }}
</pre>
</div>
</div>
</div>
</div>
<div class="taglist"><a href="https://www.vandee.art/tags.html">Tags</a>: <a href="https://www.vandee.art/tag-pkm.html">PKM</a> <a href="https://www.vandee.art/tag-github.html">Github</a> <a href="https://www.vandee.art/tag-python.html">Python</a> </div></div>
<div id="postamble" class="status"><div id="search-results"></div>
<footer>
<p>
© 2022-<script>document.write(new Date().getFullYear())</script> Vandee. All rights reserved.
</p>
<div class="social-links"></div>
</footer>
<a href="#top" aria-label="go to top" title="Go to Top (Alt + G)"
class="top-link" id="top-link" accesskey="g">
<i class="fa-solid fa-angle-up fa-2xl"></i>
</a>
<script>
var mybutton = document.getElementById('top-link');
window.onscroll = function () {
if (document.body.scrollTop > 800 || document.documentElement.scrollTop > 800) {
mybutton.style.visibility = 'visible';
mybutton.style.opacity = '1';
} else {
mybutton.style.visibility = 'hidden';
mybutton.style.opacity = '0';
}
};
</script></div>
</body>
</html>