···1818# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
1919# and can be added to the global gitignore or merged into this file. For a more nuclear
2020# option (not recommended) you can uncomment the following to ignore the entire idea folder.
2121-#.idea/
2121+.idea/
2222+.vscode/
···11+ GNU AFFERO GENERAL PUBLIC LICENSE
22+ Version 3, 19 November 2007
33+44+ Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
55+ Everyone is permitted to copy and distribute verbatim copies
66+ of this license document, but changing it is not allowed.
77+88+ Preamble
99+1010+ The GNU Affero General Public License is a free, copyleft license for
1111+software and other kinds of works, specifically designed to ensure
1212+cooperation with the community in the case of network server software.
1313+1414+ The licenses for most software and other practical works are designed
1515+to take away your freedom to share and change the works. By contrast,
1616+our General Public Licenses are intended to guarantee your freedom to
1717+share and change all versions of a program--to make sure it remains free
1818+software for all its users.
1919+2020+ When we speak of free software, we are referring to freedom, not
2121+price. Our General Public Licenses are designed to make sure that you
2222+have the freedom to distribute copies of free software (and charge for
2323+them if you wish), that you receive source code or can get it if you
2424+want it, that you can change the software or use pieces of it in new
2525+free programs, and that you know you can do these things.
2626+2727+ Developers that use our General Public Licenses protect your rights
2828+with two steps: (1) assert copyright on the software, and (2) offer
2929+you this License which gives you legal permission to copy, distribute
3030+and/or modify the software.
3131+3232+ A secondary benefit of defending all users' freedom is that
3333+improvements made in alternate versions of the program, if they
3434+receive widespread use, become available for other developers to
3535+incorporate. Many developers of free software are heartened and
3636+encouraged by the resulting cooperation. However, in the case of
3737+software used on network servers, this result may fail to come about.
3838+The GNU General Public License permits making a modified version and
3939+letting the public access it on a server without ever releasing its
4040+source code to the public.
4141+4242+ The GNU Affero General Public License is designed specifically to
4343+ensure that, in such cases, the modified source code becomes available
4444+to the community. It requires the operator of a network server to
4545+provide the source code of the modified version running there to the
4646+users of that server. Therefore, public use of a modified version, on
4747+a publicly accessible server, gives the public access to the source
4848+code of the modified version.
4949+5050+ An older license, called the Affero General Public License and
5151+published by Affero, was designed to accomplish similar goals. This is
5252+a different license, not a version of the Affero GPL, but Affero has
5353+released a new version of the Affero GPL which permits relicensing under
5454+this license.
5555+5656+ The precise terms and conditions for copying, distribution and
5757+modification follow.
5858+5959+ TERMS AND CONDITIONS
6060+6161+ 0. Definitions.
6262+6363+ "This License" refers to version 3 of the GNU Affero General Public License.
6464+6565+ "Copyright" also means copyright-like laws that apply to other kinds of
6666+works, such as semiconductor masks.
6767+6868+ "The Program" refers to any copyrightable work licensed under this
6969+License. Each licensee is addressed as "you". "Licensees" and
7070+"recipients" may be individuals or organizations.
7171+7272+ To "modify" a work means to copy from or adapt all or part of the work
7373+in a fashion requiring copyright permission, other than the making of an
7474+exact copy. The resulting work is called a "modified version" of the
7575+earlier work or a work "based on" the earlier work.
7676+7777+ A "covered work" means either the unmodified Program or a work based
7878+on the Program.
7979+8080+ To "propagate" a work means to do anything with it that, without
8181+permission, would make you directly or secondarily liable for
8282+infringement under applicable copyright law, except executing it on a
8383+computer or modifying a private copy. Propagation includes copying,
8484+distribution (with or without modification), making available to the
8585+public, and in some countries other activities as well.
8686+8787+ To "convey" a work means any kind of propagation that enables other
8888+parties to make or receive copies. Mere interaction with a user through
8989+a computer network, with no transfer of a copy, is not conveying.
9090+9191+ An interactive user interface displays "Appropriate Legal Notices"
9292+to the extent that it includes a convenient and prominently visible
9393+feature that (1) displays an appropriate copyright notice, and (2)
9494+tells the user that there is no warranty for the work (except to the
9595+extent that warranties are provided), that licensees may convey the
9696+work under this License, and how to view a copy of this License. If
9797+the interface presents a list of user commands or options, such as a
9898+menu, a prominent item in the list meets this criterion.
9999+100100+ 1. Source Code.
101101+102102+ The "source code" for a work means the preferred form of the work
103103+for making modifications to it. "Object code" means any non-source
104104+form of a work.
105105+106106+ A "Standard Interface" means an interface that either is an official
107107+standard defined by a recognized standards body, or, in the case of
108108+interfaces specified for a particular programming language, one that
109109+is widely used among developers working in that language.
110110+111111+ The "System Libraries" of an executable work include anything, other
112112+than the work as a whole, that (a) is included in the normal form of
113113+packaging a Major Component, but which is not part of that Major
114114+Component, and (b) serves only to enable use of the work with that
115115+Major Component, or to implement a Standard Interface for which an
116116+implementation is available to the public in source code form. A
117117+"Major Component", in this context, means a major essential component
118118+(kernel, window system, and so on) of the specific operating system
119119+(if any) on which the executable work runs, or a compiler used to
120120+produce the work, or an object code interpreter used to run it.
121121+122122+ The "Corresponding Source" for a work in object code form means all
123123+the source code needed to generate, install, and (for an executable
124124+work) run the object code and to modify the work, including scripts to
125125+control those activities. However, it does not include the work's
126126+System Libraries, or general-purpose tools or generally available free
127127+programs which are used unmodified in performing those activities but
128128+which are not part of the work. For example, Corresponding Source
129129+includes interface definition files associated with source files for
130130+the work, and the source code for shared libraries and dynamically
131131+linked subprograms that the work is specifically designed to require,
132132+such as by intimate data communication or control flow between those
133133+subprograms and other parts of the work.
134134+135135+ The Corresponding Source need not include anything that users
136136+can regenerate automatically from other parts of the Corresponding
137137+Source.
138138+139139+ The Corresponding Source for a work in source code form is that
140140+same work.
141141+142142+ 2. Basic Permissions.
143143+144144+ All rights granted under this License are granted for the term of
145145+copyright on the Program, and are irrevocable provided the stated
146146+conditions are met. This License explicitly affirms your unlimited
147147+permission to run the unmodified Program. The output from running a
148148+covered work is covered by this License only if the output, given its
149149+content, constitutes a covered work. This License acknowledges your
150150+rights of fair use or other equivalent, as provided by copyright law.
151151+152152+ You may make, run and propagate covered works that you do not
153153+convey, without conditions so long as your license otherwise remains
154154+in force. You may convey covered works to others for the sole purpose
155155+of having them make modifications exclusively for you, or provide you
156156+with facilities for running those works, provided that you comply with
157157+the terms of this License in conveying all material for which you do
158158+not control copyright. Those thus making or running the covered works
159159+for you must do so exclusively on your behalf, under your direction
160160+and control, on terms that prohibit them from making any copies of
161161+your copyrighted material outside their relationship with you.
162162+163163+ Conveying under any other circumstances is permitted solely under
164164+the conditions stated below. Sublicensing is not allowed; section 10
165165+makes it unnecessary.
166166+167167+ 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
168168+169169+ No covered work shall be deemed part of an effective technological
170170+measure under any applicable law fulfilling obligations under article
171171+11 of the WIPO copyright treaty adopted on 20 December 1996, or
172172+similar laws prohibiting or restricting circumvention of such
173173+measures.
174174+175175+ When you convey a covered work, you waive any legal power to forbid
176176+circumvention of technological measures to the extent such circumvention
177177+is effected by exercising rights under this License with respect to
178178+the covered work, and you disclaim any intention to limit operation or
179179+modification of the work as a means of enforcing, against the work's
180180+users, your or third parties' legal rights to forbid circumvention of
181181+technological measures.
182182+183183+ 4. Conveying Verbatim Copies.
184184+185185+ You may convey verbatim copies of the Program's source code as you
186186+receive it, in any medium, provided that you conspicuously and
187187+appropriately publish on each copy an appropriate copyright notice;
188188+keep intact all notices stating that this License and any
189189+non-permissive terms added in accord with section 7 apply to the code;
190190+keep intact all notices of the absence of any warranty; and give all
191191+recipients a copy of this License along with the Program.
192192+193193+ You may charge any price or no price for each copy that you convey,
194194+and you may offer support or warranty protection for a fee.
195195+196196+ 5. Conveying Modified Source Versions.
197197+198198+ You may convey a work based on the Program, or the modifications to
199199+produce it from the Program, in the form of source code under the
200200+terms of section 4, provided that you also meet all of these conditions:
201201+202202+ a) The work must carry prominent notices stating that you modified
203203+ it, and giving a relevant date.
204204+205205+ b) The work must carry prominent notices stating that it is
206206+ released under this License and any conditions added under section
207207+ 7. This requirement modifies the requirement in section 4 to
208208+ "keep intact all notices".
209209+210210+ c) You must license the entire work, as a whole, under this
211211+ License to anyone who comes into possession of a copy. This
212212+ License will therefore apply, along with any applicable section 7
213213+ additional terms, to the whole of the work, and all its parts,
214214+ regardless of how they are packaged. This License gives no
215215+ permission to license the work in any other way, but it does not
216216+ invalidate such permission if you have separately received it.
217217+218218+ d) If the work has interactive user interfaces, each must display
219219+ Appropriate Legal Notices; however, if the Program has interactive
220220+ interfaces that do not display Appropriate Legal Notices, your
221221+ work need not make them do so.
222222+223223+ A compilation of a covered work with other separate and independent
224224+works, which are not by their nature extensions of the covered work,
225225+and which are not combined with it such as to form a larger program,
226226+in or on a volume of a storage or distribution medium, is called an
227227+"aggregate" if the compilation and its resulting copyright are not
228228+used to limit the access or legal rights of the compilation's users
229229+beyond what the individual works permit. Inclusion of a covered work
230230+in an aggregate does not cause this License to apply to the other
231231+parts of the aggregate.
232232+233233+ 6. Conveying Non-Source Forms.
234234+235235+ You may convey a covered work in object code form under the terms
236236+of sections 4 and 5, provided that you also convey the
237237+machine-readable Corresponding Source under the terms of this License,
238238+in one of these ways:
239239+240240+ a) Convey the object code in, or embodied in, a physical product
241241+ (including a physical distribution medium), accompanied by the
242242+ Corresponding Source fixed on a durable physical medium
243243+ customarily used for software interchange.
244244+245245+ b) Convey the object code in, or embodied in, a physical product
246246+ (including a physical distribution medium), accompanied by a
247247+ written offer, valid for at least three years and valid for as
248248+ long as you offer spare parts or customer support for that product
249249+ model, to give anyone who possesses the object code either (1) a
250250+ copy of the Corresponding Source for all the software in the
251251+ product that is covered by this License, on a durable physical
252252+ medium customarily used for software interchange, for a price no
253253+ more than your reasonable cost of physically performing this
254254+ conveying of source, or (2) access to copy the
255255+ Corresponding Source from a network server at no charge.
256256+257257+ c) Convey individual copies of the object code with a copy of the
258258+ written offer to provide the Corresponding Source. This
259259+ alternative is allowed only occasionally and noncommercially, and
260260+ only if you received the object code with such an offer, in accord
261261+ with subsection 6b.
262262+263263+ d) Convey the object code by offering access from a designated
264264+ place (gratis or for a charge), and offer equivalent access to the
265265+ Corresponding Source in the same way through the same place at no
266266+ further charge. You need not require recipients to copy the
267267+ Corresponding Source along with the object code. If the place to
268268+ copy the object code is a network server, the Corresponding Source
269269+ may be on a different server (operated by you or a third party)
270270+ that supports equivalent copying facilities, provided you maintain
271271+ clear directions next to the object code saying where to find the
272272+ Corresponding Source. Regardless of what server hosts the
273273+ Corresponding Source, you remain obligated to ensure that it is
274274+ available for as long as needed to satisfy these requirements.
275275+276276+ e) Convey the object code using peer-to-peer transmission, provided
277277+ you inform other peers where the object code and Corresponding
278278+ Source of the work are being offered to the general public at no
279279+ charge under subsection 6d.
280280+281281+ A separable portion of the object code, whose source code is excluded
282282+from the Corresponding Source as a System Library, need not be
283283+included in conveying the object code work.
284284+285285+ A "User Product" is either (1) a "consumer product", which means any
286286+tangible personal property which is normally used for personal, family,
287287+or household purposes, or (2) anything designed or sold for incorporation
288288+into a dwelling. In determining whether a product is a consumer product,
289289+doubtful cases shall be resolved in favor of coverage. For a particular
290290+product received by a particular user, "normally used" refers to a
291291+typical or common use of that class of product, regardless of the status
292292+of the particular user or of the way in which the particular user
293293+actually uses, or expects or is expected to use, the product. A product
294294+is a consumer product regardless of whether the product has substantial
295295+commercial, industrial or non-consumer uses, unless such uses represent
296296+the only significant mode of use of the product.
297297+298298+ "Installation Information" for a User Product means any methods,
299299+procedures, authorization keys, or other information required to install
300300+and execute modified versions of a covered work in that User Product from
301301+a modified version of its Corresponding Source. The information must
302302+suffice to ensure that the continued functioning of the modified object
303303+code is in no case prevented or interfered with solely because
304304+modification has been made.
305305+306306+ If you convey an object code work under this section in, or with, or
307307+specifically for use in, a User Product, and the conveying occurs as
308308+part of a transaction in which the right of possession and use of the
309309+User Product is transferred to the recipient in perpetuity or for a
310310+fixed term (regardless of how the transaction is characterized), the
311311+Corresponding Source conveyed under this section must be accompanied
312312+by the Installation Information. But this requirement does not apply
313313+if neither you nor any third party retains the ability to install
314314+modified object code on the User Product (for example, the work has
315315+been installed in ROM).
316316+317317+ The requirement to provide Installation Information does not include a
318318+requirement to continue to provide support service, warranty, or updates
319319+for a work that has been modified or installed by the recipient, or for
320320+the User Product in which it has been modified or installed. Access to a
321321+network may be denied when the modification itself materially and
322322+adversely affects the operation of the network or violates the rules and
323323+protocols for communication across the network.
324324+325325+ Corresponding Source conveyed, and Installation Information provided,
326326+in accord with this section must be in a format that is publicly
327327+documented (and with an implementation available to the public in
328328+source code form), and must require no special password or key for
329329+unpacking, reading or copying.
330330+331331+ 7. Additional Terms.
332332+333333+ "Additional permissions" are terms that supplement the terms of this
334334+License by making exceptions from one or more of its conditions.
335335+Additional permissions that are applicable to the entire Program shall
336336+be treated as though they were included in this License, to the extent
337337+that they are valid under applicable law. If additional permissions
338338+apply only to part of the Program, that part may be used separately
339339+under those permissions, but the entire Program remains governed by
340340+this License without regard to the additional permissions.
341341+342342+ When you convey a copy of a covered work, you may at your option
343343+remove any additional permissions from that copy, or from any part of
344344+it. (Additional permissions may be written to require their own
345345+removal in certain cases when you modify the work.) You may place
346346+additional permissions on material, added by you to a covered work,
347347+for which you have or can give appropriate copyright permission.
348348+349349+ Notwithstanding any other provision of this License, for material you
350350+add to a covered work, you may (if authorized by the copyright holders of
351351+that material) supplement the terms of this License with terms:
352352+353353+ a) Disclaiming warranty or limiting liability differently from the
354354+ terms of sections 15 and 16 of this License; or
355355+356356+ b) Requiring preservation of specified reasonable legal notices or
357357+ author attributions in that material or in the Appropriate Legal
358358+ Notices displayed by works containing it; or
359359+360360+ c) Prohibiting misrepresentation of the origin of that material, or
361361+ requiring that modified versions of such material be marked in
362362+ reasonable ways as different from the original version; or
363363+364364+ d) Limiting the use for publicity purposes of names of licensors or
365365+ authors of the material; or
366366+367367+ e) Declining to grant rights under trademark law for use of some
368368+ trade names, trademarks, or service marks; or
369369+370370+ f) Requiring indemnification of licensors and authors of that
371371+ material by anyone who conveys the material (or modified versions of
372372+ it) with contractual assumptions of liability to the recipient, for
373373+ any liability that these contractual assumptions directly impose on
374374+ those licensors and authors.
375375+376376+ All other non-permissive additional terms are considered "further
377377+restrictions" within the meaning of section 10. If the Program as you
378378+received it, or any part of it, contains a notice stating that it is
379379+governed by this License along with a term that is a further
380380+restriction, you may remove that term. If a license document contains
381381+a further restriction but permits relicensing or conveying under this
382382+License, you may add to a covered work material governed by the terms
383383+of that license document, provided that the further restriction does
384384+not survive such relicensing or conveying.
385385+386386+ If you add terms to a covered work in accord with this section, you
387387+must place, in the relevant source files, a statement of the
388388+additional terms that apply to those files, or a notice indicating
389389+where to find the applicable terms.
390390+391391+ Additional terms, permissive or non-permissive, may be stated in the
392392+form of a separately written license, or stated as exceptions;
393393+the above requirements apply either way.
394394+395395+ 8. Termination.
396396+397397+ You may not propagate or modify a covered work except as expressly
398398+provided under this License. Any attempt otherwise to propagate or
399399+modify it is void, and will automatically terminate your rights under
400400+this License (including any patent licenses granted under the third
401401+paragraph of section 11).
402402+403403+ However, if you cease all violation of this License, then your
404404+license from a particular copyright holder is reinstated (a)
405405+provisionally, unless and until the copyright holder explicitly and
406406+finally terminates your license, and (b) permanently, if the copyright
407407+holder fails to notify you of the violation by some reasonable means
408408+prior to 60 days after the cessation.
409409+410410+ Moreover, your license from a particular copyright holder is
411411+reinstated permanently if the copyright holder notifies you of the
412412+violation by some reasonable means, this is the first time you have
413413+received notice of violation of this License (for any work) from that
414414+copyright holder, and you cure the violation prior to 30 days after
415415+your receipt of the notice.
416416+417417+ Termination of your rights under this section does not terminate the
418418+licenses of parties who have received copies or rights from you under
419419+this License. If your rights have been terminated and not permanently
420420+reinstated, you do not qualify to receive new licenses for the same
421421+material under section 10.
422422+423423+ 9. Acceptance Not Required for Having Copies.
424424+425425+ You are not required to accept this License in order to receive or
426426+run a copy of the Program. Ancillary propagation of a covered work
427427+occurring solely as a consequence of using peer-to-peer transmission
428428+to receive a copy likewise does not require acceptance. However,
429429+nothing other than this License grants you permission to propagate or
430430+modify any covered work. These actions infringe copyright if you do
431431+not accept this License. Therefore, by modifying or propagating a
432432+covered work, you indicate your acceptance of this License to do so.
433433+434434+ 10. Automatic Licensing of Downstream Recipients.
435435+436436+ Each time you convey a covered work, the recipient automatically
437437+receives a license from the original licensors, to run, modify and
438438+propagate that work, subject to this License. You are not responsible
439439+for enforcing compliance by third parties with this License.
440440+441441+ An "entity transaction" is a transaction transferring control of an
442442+organization, or substantially all assets of one, or subdividing an
443443+organization, or merging organizations. If propagation of a covered
444444+work results from an entity transaction, each party to that
445445+transaction who receives a copy of the work also receives whatever
446446+licenses to the work the party's predecessor in interest had or could
447447+give under the previous paragraph, plus a right to possession of the
448448+Corresponding Source of the work from the predecessor in interest, if
449449+the predecessor has it or can get it with reasonable efforts.
450450+451451+ You may not impose any further restrictions on the exercise of the
452452+rights granted or affirmed under this License. For example, you may
453453+not impose a license fee, royalty, or other charge for exercise of
454454+rights granted under this License, and you may not initiate litigation
455455+(including a cross-claim or counterclaim in a lawsuit) alleging that
456456+any patent claim is infringed by making, using, selling, offering for
457457+sale, or importing the Program or any portion of it.
458458+459459+ 11. Patents.
460460+461461+ A "contributor" is a copyright holder who authorizes use under this
462462+License of the Program or a work on which the Program is based. The
463463+work thus licensed is called the contributor's "contributor version".
464464+465465+ A contributor's "essential patent claims" are all patent claims
466466+owned or controlled by the contributor, whether already acquired or
467467+hereafter acquired, that would be infringed by some manner, permitted
468468+by this License, of making, using, or selling its contributor version,
469469+but do not include claims that would be infringed only as a
470470+consequence of further modification of the contributor version. For
471471+purposes of this definition, "control" includes the right to grant
472472+patent sublicenses in a manner consistent with the requirements of
473473+this License.
474474+475475+ Each contributor grants you a non-exclusive, worldwide, royalty-free
476476+patent license under the contributor's essential patent claims, to
477477+make, use, sell, offer for sale, import and otherwise run, modify and
478478+propagate the contents of its contributor version.
479479+480480+ In the following three paragraphs, a "patent license" is any express
481481+agreement or commitment, however denominated, not to enforce a patent
482482+(such as an express permission to practice a patent or covenant not to
483483+sue for patent infringement). To "grant" such a patent license to a
484484+party means to make such an agreement or commitment not to enforce a
485485+patent against the party.
486486+487487+ If you convey a covered work, knowingly relying on a patent license,
488488+and the Corresponding Source of the work is not available for anyone
489489+to copy, free of charge and under the terms of this License, through a
490490+publicly available network server or other readily accessible means,
491491+then you must either (1) cause the Corresponding Source to be so
492492+available, or (2) arrange to deprive yourself of the benefit of the
493493+patent license for this particular work, or (3) arrange, in a manner
494494+consistent with the requirements of this License, to extend the patent
495495+license to downstream recipients. "Knowingly relying" means you have
496496+actual knowledge that, but for the patent license, your conveying the
497497+covered work in a country, or your recipient's use of the covered work
498498+in a country, would infringe one or more identifiable patents in that
499499+country that you have reason to believe are valid.
500500+501501+ If, pursuant to or in connection with a single transaction or
502502+arrangement, you convey, or propagate by procuring conveyance of, a
503503+covered work, and grant a patent license to some of the parties
504504+receiving the covered work authorizing them to use, propagate, modify
505505+or convey a specific copy of the covered work, then the patent license
506506+you grant is automatically extended to all recipients of the covered
507507+work and works based on it.
508508+509509+ A patent license is "discriminatory" if it does not include within
510510+the scope of its coverage, prohibits the exercise of, or is
511511+conditioned on the non-exercise of one or more of the rights that are
512512+specifically granted under this License. You may not convey a covered
513513+work if you are a party to an arrangement with a third party that is
514514+in the business of distributing software, under which you make payment
515515+to the third party based on the extent of your activity of conveying
516516+the work, and under which the third party grants, to any of the
517517+parties who would receive the covered work from you, a discriminatory
518518+patent license (a) in connection with copies of the covered work
519519+conveyed by you (or copies made from those copies), or (b) primarily
520520+for and in connection with specific products or compilations that
521521+contain the covered work, unless you entered into that arrangement,
522522+or that patent license was granted, prior to 28 March 2007.
523523+524524+ Nothing in this License shall be construed as excluding or limiting
525525+any implied license or other defenses to infringement that may
526526+otherwise be available to you under applicable patent law.
527527+528528+ 12. No Surrender of Others' Freedom.
529529+530530+ If conditions are imposed on you (whether by court order, agreement or
531531+otherwise) that contradict the conditions of this License, they do not
532532+excuse you from the conditions of this License. If you cannot convey a
533533+covered work so as to satisfy simultaneously your obligations under this
534534+License and any other pertinent obligations, then as a consequence you may
535535+not convey it at all. For example, if you agree to terms that obligate you
536536+to collect a royalty for further conveying from those to whom you convey
537537+the Program, the only way you could satisfy both those terms and this
538538+License would be to refrain entirely from conveying the Program.
539539+540540+ 13. Remote Network Interaction; Use with the GNU General Public License.
541541+542542+ Notwithstanding any other provision of this License, if you modify the
543543+Program, your modified version must prominently offer all users
544544+interacting with it remotely through a computer network (if your version
545545+supports such interaction) an opportunity to receive the Corresponding
546546+Source of your version by providing access to the Corresponding Source
547547+from a network server at no charge, through some standard or customary
548548+means of facilitating copying of software. This Corresponding Source
549549+shall include the Corresponding Source for any work covered by version 3
550550+of the GNU General Public License that is incorporated pursuant to the
551551+following paragraph.
552552+553553+ Notwithstanding any other provision of this License, you have
554554+permission to link or combine any covered work with a work licensed
555555+under version 3 of the GNU General Public License into a single
556556+combined work, and to convey the resulting work. The terms of this
557557+License will continue to apply to the part which is the covered work,
558558+but the work with which it is combined will remain governed by version
559559+3 of the GNU General Public License.
560560+561561+ 14. Revised Versions of this License.
562562+563563+ The Free Software Foundation may publish revised and/or new versions of
564564+the GNU Affero General Public License from time to time. Such new versions
565565+will be similar in spirit to the present version, but may differ in detail to
566566+address new problems or concerns.
567567+568568+ Each version is given a distinguishing version number. If the
569569+Program specifies that a certain numbered version of the GNU Affero General
570570+Public License "or any later version" applies to it, you have the
571571+option of following the terms and conditions either of that numbered
572572+version or of any later version published by the Free Software
573573+Foundation. If the Program does not specify a version number of the
574574+GNU Affero General Public License, you may choose any version ever published
575575+by the Free Software Foundation.
576576+577577+ If the Program specifies that a proxy can decide which future
578578+versions of the GNU Affero General Public License can be used, that proxy's
579579+public statement of acceptance of a version permanently authorizes you
580580+to choose that version for the Program.
581581+582582+ Later license versions may give you additional or different
583583+permissions. However, no additional obligations are imposed on any
584584+author or copyright holder as a result of your choosing to follow a
585585+later version.
586586+587587+ 15. Disclaimer of Warranty.
588588+589589+ THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
590590+APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
591591+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
592592+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
593593+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
594594+PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
595595+IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
596596+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
597597+598598+ 16. Limitation of Liability.
599599+600600+ IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
601601+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
602602+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
603603+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
604604+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
605605+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
606606+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
607607+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
608608+SUCH DAMAGES.
609609+610610+ 17. Interpretation of Sections 15 and 16.
611611+612612+ If the disclaimer of warranty and limitation of liability provided
613613+above cannot be given local legal effect according to their terms,
614614+reviewing courts shall apply local law that most closely approximates
615615+an absolute waiver of all civil liability in connection with the
616616+Program, unless a warranty or assumption of liability accompanies a
617617+copy of the Program in return for a fee.
618618+619619+ END OF TERMS AND CONDITIONS
620620+621621+ How to Apply These Terms to Your New Programs
622622+623623+ If you develop a new program, and you want it to be of the greatest
624624+possible use to the public, the best way to achieve this is to make it
625625+free software which everyone can redistribute and change under these terms.
626626+627627+ To do so, attach the following notices to the program. It is safest
628628+to attach them to the start of each source file to most effectively
629629+state the exclusion of warranty; and each file should have at least
630630+the "copyright" line and a pointer to where the full notice is found.
631631+632632+ <one line to give the program's name and a brief idea of what it does.>
633633+ Copyright (C) 2025 <name of author>
634634+635635+ This program is free software: you can redistribute it and/or modify
636636+ it under the terms of the GNU Affero General Public License as published
637637+ by the Free Software Foundation, either version 3 of the License, or
638638+ (at your option) any later version.
639639+640640+ This program is distributed in the hope that it will be useful,
641641+ but WITHOUT ANY WARRANTY; without even the implied warranty of
642642+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
643643+ GNU Affero General Public License for more details.
644644+645645+ You should have received a copy of the GNU Affero General Public License
646646+ along with this program. If not, see <https://www.gnu.org/licenses/>.
647647+648648+Also add information on how to contact you by electronic and paper mail.
649649+650650+ If your software can interact with users remotely through a computer
651651+network, you should also make sure that it provides a way for users to
652652+get its source. For example, if your program is a web application, its
653653+interface could display a "Source" link that leads users to an archive
654654+of the code. There are many ways you could offer source, and different
655655+solutions will be better for different programs; see section 13 for the
656656+specific requirements.
657657+658658+ You should also get your employer (if you work as a programmer) or school,
659659+if any, to sign a "copyright disclaimer" for the program, if necessary.
660660+For more information on this, and how to apply and follow the GNU AGPL, see
661661+<https://www.gnu.org/licenses/>.
+207
README.md
···11+<!-- markdownlint-disable MD033 -->
22+13# Personal Activity Index
2435A CLI that ingests content from Substack, Bluesky, and Leaflet into SQLite, with an optional Cloudflare Worker + D1 deployment path.
66+77+## Features
88+99+- Fetch posts from multiple sources:
1010+ - **Substack** via RSS feeds
1111+ - **Bluesky** via AT Protocol
1212+ - **Leaflet** publications via RSS feeds
1313+- Local SQLite storage with full-text search
1414+- Flexible filtering and querying
1515+- Self-hostable or serverless (Cloudflare Workers)
1616+1717+## Quick Start
1818+1919+```bash
2020+# Install
2121+cargo install --path cli
2222+2323+# Initialize config (creates ~/.config/pai/config.toml)
2424+pai init
2525+2626+# Edit config with your sources
2727+$EDITOR ~/.config/pai/config.toml
2828+2929+# Sync content
3030+pai sync
3131+3232+# List items
3333+pai list -n 10
3434+3535+# Check database
3636+pai db-check
3737+```
3838+3939+## Configuration
4040+4141+Configuration is loaded from `$XDG_CONFIG_HOME/pai/config.toml` or `$HOME/.config/pai/config.toml`.
4242+4343+See [config.example.toml](./config.example.toml) for a complete example with all available options.
4444+4545+## Architecture
4646+4747+The project is organized as a Cargo workspace
4848+4949+```sh
5050+.
5151+├── core # Shared types, fetchers, and the storage trait
5252+├── cli # CLI binary (POSIX-compliant)
5353+└── worker # Cloudflare Worker deployment using workers-rs
5454+```
5555+5656+<details>
5757+<summary><strong>Source Implementations</strong></summary>
5858+5959+### Substack (RSS)
6060+6161+Substack fetcher uses standard RSS 2.0 feeds available at `{base_url}/feed`.
6262+6363+**Implementation:**
6464+6565+- Fetches RSS feed using `feed-rs` parser
6666+- Maps RSS `<item>` elements to standardized `Item` struct
6767+- Uses GUID as item ID, falls back to link if GUID is missing
6868+- Normalizes `pubDate` to ISO 8601 format
6969+7070+**Key mappings:**
7171+7272+- `id` = RSS GUID or link
7373+- `source_kind` = `substack`
7474+- `source_id` = Domain extracted from base_url
7575+- `title` = RSS title
7676+- `summary` = RSS description
7777+- `url` = RSS link
7878+- `content_html` = RSS content (if available)
7979+- `published_at` = RSS pubDate (normalized to ISO 8601)
8080+8181+**Example RSS structure:**
8282+8383+```xml
8484+<item>
8585+ <title>Post Title</title>
8686+ <link>https://example.substack.com/p/post-slug</link>
8787+ <guid>https://example.substack.com/p/post-slug</guid>
8888+ <pubDate>Mon, 01 Jan 2024 12:00:00 +0000</pubDate>
8989+ <description>Post summary or excerpt</description>
9090+</item>
9191+```
9292+9393+### AT Protocol Integration (Bluesky)
9494+9595+#### Overview
9696+9797+Bluesky is built on the AT Protocol (Authenticated Transfer Protocol), a decentralized social networking protocol.
9898+9999+**Key Concepts:**
100100+101101+- **DID (Decentralized Identifier)**: Unique identifier for users (e.g., `did:plc:xyz123`)
102102+- **Handle**: Human-readable identifier (e.g., `user.bsky.social`)
103103+- **AT URI**: Resource identifier (e.g., `at://did:plc:xyz/app.bsky.feed.post/abc123`)
104104+- **Lexicon**: Schema definition language for records and API methods
105105+- **XRPC**: HTTP API wrapper for AT Protocol methods
106106+- **PDS (Personal Data Server)**: Server that stores user data
107107+108108+#### Implementation
109109+110110+Bluesky uses standard `app.bsky.feed.post` records and provides a public API for fetching posts.
111111+112112+**Endpoint:** `GET https://public.api.bsky.app/xrpc/app.bsky.feed.getAuthorFeed`
113113+114114+**Parameters:**
115115+116116+- `actor` - User handle or DID
117117+- `limit` - Number of posts to fetch (default: 50)
118118+- `cursor` - Pagination cursor (optional)
119119+120120+**Implementation:**
121121+122122+- Fetches author feed using `app.bsky.feed.getAuthorFeed`
123123+- Filters out reposts and quotes (only includes original posts)
124124+- Converts AT URIs to canonical Bluesky URLs
125125+- Truncates long post text to create titles
126126+127127+**Key mappings:**
128128+129129+- `id` = AT URI (e.g., `at://did:plc:xyz/app.bsky.feed.post/abc123`)
130130+- `source_kind` = `bluesky`
131131+- `source_id` = User handle
132132+- `title` = Truncated post text (first 100 chars)
133133+- `summary` = Full post text
134134+- `url` = Canonical URL (`https://bsky.app/profile/{handle}/post/{post_id}`)
135135+- `author` = Post author handle
136136+- `published_at` = Post `createdAt` timestamp
137137+138138+**Filtering reposts:**
139139+Posts with a `reason` field (indicating repost or quote) are excluded to fetch only original content.
140140+141141+### Leaflet (RSS)
142142+143143+#### Overview
144144+145145+Leaflet publications provide RSS feeds at `{base_url}/rss`, making them straightforward to fetch using standard RSS parsing.
146146+147147+**Note:** While Leaflet is built on AT Protocol and uses custom `pub.leaflet.post` records, we use RSS feeds for simplicity and reliability. Leaflet's RSS implementation provides all necessary metadata without requiring AT Protocol PDS queries.
148148+149149+**Implementation:**
150150+151151+- Fetches RSS feed using `feed-rs` parser
152152+- Maps RSS `<item>` elements to standardized `Item` struct
153153+- Supports multiple publications via config array
154154+- Uses entry ID from feed, falls back to link if missing
155155+- Normalizes publication dates to ISO 8601 format
156156+157157+**Key mappings:**
158158+159159+- `id` = RSS entry ID or link
160160+- `source_kind` = `leaflet`
161161+- `source_id` = Publication ID from config (e.g., `desertthunder`, `stormlightlabs`)
162162+- `title` = RSS entry title
163163+- `summary` = RSS entry summary/description
164164+- `url` = RSS entry link
165165+- `content_html` = RSS content body (if available)
166166+- `author` = RSS entry author
167167+- `published_at` = RSS published date or updated date (normalized to ISO 8601)
168168+169169+**Configuration:**
170170+171171+Leaflet supports multiple publications through array configuration:
172172+173173+```toml
174174+[[sources.leaflet]]
175175+enabled = true
176176+id = "desertthunder"
177177+base_url = "https://desertthunder.leaflet.pub"
178178+179179+[[sources.leaflet]]
180180+enabled = true
181181+id = "stormlightlabs"
182182+base_url = "https://stormlightlabs.leaflet.pub"
183183+```
184184+185185+**Example RSS structure:**
186186+187187+```xml
188188+<item>
189189+ <title>Dev Log: 2025-11-22</title>
190190+ <link>https://desertthunder.leaflet.pub/3m6a7fuk7u22p</link>
191191+ <guid>https://desertthunder.leaflet.pub/3m6a7fuk7u22p</guid>
192192+ <pubDate>Fri, 22 Nov 2025 16:22:54 +0000</pubDate>
193193+ <description>Post summary or excerpt</description>
194194+</item>
195195+```
196196+197197+</details>
198198+199199+## References
200200+201201+- [AT Protocol Documentation](https://atproto.com)
202202+- [Lexicon Guide](https://atproto.com/guides/lexicon) - Schema definition language
203203+- [XRPC Specification](https://atproto.com/specs/xrpc) - HTTP API wrapper
204204+- [Bluesky API Documentation](https://docs.bsky.app/)
205205+- [Leaflet](https://tangled.org/leaflet.pub/leaflet) - Leaflet source code
206206+- [Leaflet Manual](https://about.leaflet.pub/) - User-facing documentation
207207+208208+## License
209209+210210+See [LICENSE file](./LICENSE) for details.
+1
cli/Cargo.toml
···1313rusqlite = { version = "0.37", features = ["bundled"] }
1414chrono = "0.4"
1515dirs = "6.0"
1616+owo-colors = "4.1"
+138-58
cli/src/main.rs
···22mod storage;
3344use clap::{Parser, Subcommand};
55+use owo_colors::OwoColorize;
56use pai_core::{Config, ListFilter, PaiError, SourceKind};
67use std::path::PathBuf;
78use storage::SqliteStorage;
···2324 command: Commands,
2425}
25262727+#[derive(Parser, Debug)]
2828+struct ExportOpts {
2929+ /// Filter by source kind
3030+ #[arg(short = 'k', value_name = "KIND")]
3131+ kind: Option<SourceKind>,
3232+3333+ /// Filter by specific source ID
3434+ #[arg(short = 'S', value_name = "ID")]
3535+ source_id: Option<String>,
3636+3737+ /// Maximum number of items
3838+ #[arg(short = 'n', value_name = "NUMBER")]
3939+ limit: Option<usize>,
4040+4141+ /// Only items published at or after this time
4242+ #[arg(short = 's', value_name = "TIME")]
4343+ since: Option<String>,
4444+4545+ /// Filter items by substring
4646+ #[arg(short = 'q', value_name = "PATTERN")]
4747+ query: Option<String>,
4848+4949+ /// Output format
5050+ #[arg(short = 'f', value_name = "FORMAT", default_value = "json")]
5151+ format: String,
5252+5353+ /// Output file (default: stdout)
5454+ #[arg(short = 'o', value_name = "FILE")]
5555+ output: Option<PathBuf>,
5656+}
5757+5858+impl From<ExportOpts> for ListFilter {
5959+ fn from(opts: ExportOpts) -> Self {
6060+ ListFilter {
6161+ source_kind: opts.kind,
6262+ source_id: opts.source_id,
6363+ limit: opts.limit,
6464+ since: opts.since,
6565+ query: opts.query,
6666+ }
6767+ }
6868+}
6969+2670#[derive(Subcommand, Debug)]
2771enum Commands {
2872 /// Fetch and store content from configured sources
···64108 },
6510966110 /// Produce feeds or export files
6767- Export {
6868- /// Filter by source kind
6969- #[arg(short = 'k', value_name = "KIND")]
7070- kind: Option<SourceKind>,
7171-7272- /// Filter by specific source ID
7373- #[arg(short = 'S', value_name = "ID")]
7474- source_id: Option<String>,
7575-7676- /// Maximum number of items
7777- #[arg(short = 'n', value_name = "NUMBER")]
7878- limit: Option<usize>,
7979-8080- /// Only items published at or after this time
8181- #[arg(short = 's', value_name = "TIME")]
8282- since: Option<String>,
8383-8484- /// Filter items by substring
8585- #[arg(short = 'q', value_name = "PATTERN")]
8686- query: Option<String>,
8787-8888- /// Output format
8989- #[arg(short = 'f', value_name = "FORMAT", default_value = "json")]
9090- format: String,
9191-9292- /// Output file (default: stdout)
9393- #[arg(short = 'o', value_name = "FILE")]
9494- output: Option<PathBuf>,
9595- },
111111+ Export(ExportOpts),
9611297113 /// Self-host HTTP API
98114 Serve {
···103119104120 /// Verify database schema and print statistics
105121 DbCheck,
122122+123123+ /// Initialize configuration file
124124+ Init {
125125+ /// Force overwrite existing config
126126+ #[arg(short = 'f')]
127127+ force: bool,
128128+ },
106129}
107130108131fn main() {
···113136 Commands::List { kind, source_id, limit, since, query } => {
114137 handle_list(cli.db_path, kind, source_id, limit, since, query)
115138 }
116116- Commands::Export { kind, source_id, limit, since, query, format, output } => {
117117- handle_export(cli.db_path, kind, source_id, limit, since, query, format, output)
118118- }
139139+ Commands::Export(opts) => handle_export(cli.db_path, opts),
119140 Commands::Serve { address } => handle_serve(cli.db_path, address),
120141 Commands::DbCheck => handle_db_check(cli.db_path),
142142+ Commands::Init { force } => handle_init(cli.config_dir, force),
121143 };
122144123145 if let Err(e) = result {
124124- eprintln!("Error: {e}");
146146+ eprintln!("{} {}", "Error:".red().bold(), e);
125147 std::process::exit(1);
126148 }
127149}
128150129151fn handle_sync(
130130- config_dir: Option<PathBuf>, db_path: Option<PathBuf>, _all: bool, _kind: Option<SourceKind>,
131131- _source_id: Option<String>,
152152+ config_dir: Option<PathBuf>, db_path: Option<PathBuf>, _all: bool, kind: Option<SourceKind>,
153153+ source_id: Option<String>,
132154) -> Result<(), PaiError> {
133155 let db_path = paths::resolve_db_path(db_path)?;
134134- let _config_dir = paths::resolve_config_dir(config_dir)?;
156156+ let config_dir = paths::resolve_config_dir(config_dir)?;
135157136158 let storage = SqliteStorage::new(db_path)?;
137137- let config = Config::default();
138159139139- let count = pai_core::sync_all_sources(&config, &storage)?;
160160+ let config_path = config_dir.join("config.toml");
161161+ let config = if config_path.exists() {
162162+ Config::from_file(&config_path)?
163163+ } else {
164164+ println!(
165165+ "{} No config file found, using default configuration",
166166+ "Warning:".yellow()
167167+ );
168168+ Config::default()
169169+ };
140170141141- println!("Synced {count} items");
171171+ let count = pai_core::sync_all_sources(&config, &storage, kind, source_id.as_deref())?;
172172+173173+ if count == 0 {
174174+ println!("{} No sources synced (check your config or filters)", "Info:".cyan());
175175+ } else {
176176+ println!("{} Synced {}", "Success:".green(), format!("{count} source(s)").bold());
177177+ }
178178+142179 Ok(())
143180}
144181···154191 let items = pai_core::Storage::list_items(&storage, &filter)?;
155192156193 if items.is_empty() {
157157- println!("No items found");
194194+ println!("{}", "No items found".yellow());
158195 return Ok(());
159196 }
160197161161- println!("Found {} items:\n", items.len());
198198+ println!("{} {}\n", "Found".cyan(), format!("{} items:", items.len()).bold());
162199 for item in items {
163163- println!("ID: {}", item.id);
164164- println!("Source: {} ({})", item.source_kind, item.source_id);
200200+ println!("{} {}", "ID:".bright_black(), item.id);
201201+ println!(
202202+ "{} {} {}",
203203+ "Source:".bright_black(),
204204+ item.source_kind.to_string().cyan(),
205205+ format!("({})", item.source_id).bright_black()
206206+ );
165207 if let Some(title) = &item.title {
166166- println!("Title: {title}");
208208+ println!("{} {}", "Title:".bright_black(), title.bold());
167209 }
168210 if let Some(author) = &item.author {
169169- println!("Author: {author}");
211211+ println!("{} {}", "Author:".bright_black(), author);
170212 }
171171- println!("URL: {}", item.url);
172172- println!("Published: {}", item.published_at);
213213+ println!("{} {}", "URL:".bright_black(), item.url.blue().underline());
214214+ println!("{} {}", "Published:".bright_black(), item.published_at);
173215 println!();
174216 }
175217176218 Ok(())
177219}
178220179179-fn handle_export(
180180- db_path: Option<PathBuf>, kind: Option<SourceKind>, source_id: Option<String>, limit: Option<usize>,
181181- since: Option<String>, query: Option<String>, format: String, output: Option<PathBuf>,
182182-) -> Result<(), PaiError> {
221221+fn handle_export(db_path: Option<PathBuf>, opts: ExportOpts) -> Result<(), PaiError> {
183222 let db_path = paths::resolve_db_path(db_path)?;
184223 let _storage = SqliteStorage::new(db_path)?;
185224186186- let filter = ListFilter { source_kind: kind, source_id, limit, since, query };
225225+ let format = opts.format.clone();
226226+ let output = opts.output.clone();
227227+ let filter: ListFilter = opts.into();
187228188229 println!("export command - format: {format}, output: {output:?}, filter: {filter:?}");
189230 Ok(())
···201242 let db_path = paths::resolve_db_path(db_path)?;
202243 let storage = SqliteStorage::new(db_path)?;
203244204204- println!("Verifying database schema...");
245245+ println!("{}", "Verifying database schema...".cyan());
205246 storage.verify_schema()?;
206206- println!("Schema verification: OK\n");
247247+ println!("{} {}\n", "Schema verification:".green(), "OK".bold());
207248208208- println!("Database statistics:");
249249+ println!("{}", "Database statistics:".cyan().bold());
209250 let total = storage.count_items()?;
210210- println!(" Total items: {total}");
251251+ println!(" {}: {}", "Total items".bright_black(), total.to_string().bold());
211252212253 let stats = storage.get_stats()?;
213254 if !stats.is_empty() {
214214- println!("\nItems by source:");
255255+ println!("\n{}", "Items by source:".cyan().bold());
215256 for (source_kind, count) in stats {
216216- println!(" {source_kind}: {count}");
257257+ println!(" {}: {}", source_kind.bright_black(), count.to_string().bold());
217258 }
218259 }
219260220261 Ok(())
221262}
263263+264264+fn handle_init(config_dir: Option<PathBuf>, force: bool) -> Result<(), PaiError> {
265265+ let config_dir = paths::resolve_config_dir(config_dir)?;
266266+ let config_path = config_dir.join("config.toml");
267267+268268+ if config_path.exists() && !force {
269269+ println!(
270270+ "{} Config file already exists at {}",
271271+ "Error:".red().bold(),
272272+ config_path.display()
273273+ );
274274+ println!("{} Use {} to overwrite", "Hint:".yellow(), "pai init -f".bold());
275275+ return Err(PaiError::Config("Config file already exists".to_string()));
276276+ }
277277+278278+ std::fs::create_dir_all(&config_dir)
279279+ .map_err(|e| PaiError::Config(format!("Failed to create config directory: {e}")))?;
280280+281281+ let default_config = include_str!("../../config.example.toml");
282282+ std::fs::write(&config_path, default_config)
283283+ .map_err(|e| PaiError::Config(format!("Failed to write config file: {e}")))?;
284284+285285+ println!("{} Created configuration file", "Success:".green().bold());
286286+ println!(
287287+ " {}: {}",
288288+ "Location".bright_black(),
289289+ config_path.display().to_string().bold()
290290+ );
291291+ println!();
292292+ println!("{}", "Next steps:".cyan().bold());
293293+ println!(" 1. Edit the config file to add your sources:");
294294+ println!(" {}", format!("$EDITOR {}", config_path.display()).bright_black());
295295+ println!(" 2. Run sync to fetch content:");
296296+ println!(" {}", "pai sync".bright_black());
297297+ println!(" 3. List your items:");
298298+ println!(" {}", "pai list -n 10".bright_black());
299299+300300+ Ok(())
301301+}
+41
config.example.toml
···11+# Personal Activity Index Configuration Example
22+# Copy this file to your config directory and customize as needed
33+#
44+# Default config location:
55+# - $XDG_CONFIG_HOME/pai/config.toml
66+# - $HOME/.config/pai/config.toml
77+88+[database]
99+# Path to SQLite database file (optional, defaults to $XDG_DATA_HOME/pai/pai.db)
1010+path = "/home/owais/.local/share/pai/pai.db"
1111+1212+[deployment]
1313+# Deployment mode: "sqlite" for local, "cloudflare" for Workers
1414+mode = "sqlite"
1515+1616+# Cloudflare deployment configuration (optional, only needed for Workers)
1717+[deployment.cloudflare]
1818+worker_name = "personal-activity-index"
1919+d1_binding = "DB"
2020+database_name = "personal_activity_db"
2121+2222+# Substack RSS feed source
2323+[sources.substack]
2424+enabled = true
2525+base_url = "https://patternmatched.substack.com"
2626+2727+# Bluesky AT Protocol source
2828+[sources.bluesky]
2929+enabled = true
3030+handle = "desertthunder.dev"
3131+3232+# Leaflet publications (can have multiple)
3333+[[sources.leaflet]]
3434+enabled = true
3535+id = "desertthunder"
3636+base_url = "https://desertthunder.leaflet.pub"
3737+3838+[[sources.leaflet]]
3939+enabled = true
4040+id = "stormlightlabs"
4141+base_url = "https://stormlightlabs.leaflet.pub"
+7
core/Cargo.toml
···5566[dependencies]
77thiserror = "2.0.17"
88+serde = { version = "1.0", features = ["derive"] }
99+serde_json = "1.0"
1010+toml = "0.9"
1111+reqwest = { version = "0.12", features = ["json"] }
1212+tokio = { version = "1.0", features = ["rt-multi-thread", "macros"] }
1313+feed-rs = "2.2"
1414+chrono = "0.4"
+251
core/src/fetchers/bluesky.rs
···11+use crate::{BlueskyConfig, Item, PaiError, Result, SourceFetcher, SourceKind, Storage};
22+use chrono::Utc;
33+use serde::Deserialize;
44+55+const BLUESKY_API_BASE: &str = "https://public.api.bsky.app";
66+77+/// Response from app.bsky.feed.getAuthorFeed
88+#[derive(Debug, Deserialize)]
99+struct AuthorFeedResponse {
1010+ feed: Vec<FeedViewPost>,
1111+ #[allow(dead_code)]
1212+ cursor: Option<String>,
1313+}
1414+1515+/// A post in the author feed
1616+#[derive(Debug, Deserialize)]
1717+struct FeedViewPost {
1818+ post: PostView,
1919+ #[allow(dead_code)]
2020+ reason: Option<serde_json::Value>,
2121+}
2222+2323+/// Post view with metadata
2424+#[derive(Debug, Deserialize)]
2525+struct PostView {
2626+ uri: String,
2727+ #[allow(dead_code)]
2828+ cid: String,
2929+ author: Author,
3030+ record: serde_json::Value,
3131+ #[allow(dead_code)]
3232+ #[serde(rename = "indexedAt")]
3333+ indexed_at: String,
3434+}
3535+3636+/// Author information
3737+#[derive(Debug, Deserialize)]
3838+struct Author {
3939+ #[allow(dead_code)]
4040+ did: String,
4141+ handle: String,
4242+ #[allow(dead_code)]
4343+ #[serde(rename = "displayName")]
4444+ display_name: Option<String>,
4545+}
4646+4747+/// Fetcher for Bluesky posts via AT Protocol
4848+///
4949+/// Retrieves posts from a Bluesky user by querying the public API.
5050+/// Filters out reposts and quotes to only include original posts.
5151+pub struct BlueskyFetcher {
5252+ config: BlueskyConfig,
5353+ client: reqwest::Client,
5454+}
5555+5656+impl BlueskyFetcher {
5757+ /// Creates a new Bluesky fetcher with the given configuration
5858+ pub fn new(config: BlueskyConfig) -> Self {
5959+ Self { config, client: reqwest::Client::new() }
6060+ }
6161+6262+ /// Fetches the author feed from the Bluesky public API
6363+ async fn fetch_author_feed(&self) -> Result<AuthorFeedResponse> {
6464+ let url = format!("{BLUESKY_API_BASE}/xrpc/app.bsky.feed.getAuthorFeed");
6565+6666+ let response = self
6767+ .client
6868+ .get(&url)
6969+ .query(&[("actor", &self.config.handle), ("limit", &"50".to_string())])
7070+ .send()
7171+ .await
7272+ .map_err(|e| PaiError::Fetch(format!("Failed to fetch Bluesky feed: {e}")))?;
7373+7474+ if !response.status().is_success() {
7575+ return Err(PaiError::Fetch(format!("Bluesky API error: {}", response.status())));
7676+ }
7777+7878+ response
7979+ .json::<AuthorFeedResponse>()
8080+ .await
8181+ .map_err(|e| PaiError::Parse(format!("Failed to parse Bluesky response: {e}")))
8282+ }
8383+8484+ /// Checks if a post is an original post (not a repost or quote)
8585+ fn is_original_post(feed_post: &FeedViewPost) -> bool {
8686+ feed_post.reason.is_none()
8787+ }
8888+8989+ /// Converts an AT URI to a canonical Bluesky URL
9090+ ///
9191+ /// AT URI format: at://did:plc:xyz/app.bsky.feed.post/abc123
9292+ /// URL format: https://bsky.app/profile/{handle}/post/{post_id}
9393+ fn at_uri_to_url(uri: &str, handle: &str) -> Result<String> {
9494+ let parts: Vec<&str> = uri.split('/').collect();
9595+ if parts.len() >= 4 && parts[0] == "at:" {
9696+ let post_id = parts[parts.len() - 1];
9797+ Ok(format!("https://bsky.app/profile/{handle}/post/{post_id}"))
9898+ } else {
9999+ Err(PaiError::Parse(format!("Invalid AT URI: {uri}")))
100100+ }
101101+ }
102102+103103+ /// Extracts text content from the post record
104104+ fn extract_text(record: &serde_json::Value) -> Option<String> {
105105+ record.get("text").and_then(|v| v.as_str()).map(String::from)
106106+ }
107107+108108+ /// Creates a title from the post text (truncated to 100 chars)
109109+ fn create_title(text: &str) -> String {
110110+ if text.len() <= 100 {
111111+ text.to_string()
112112+ } else {
113113+ format!("{}...", &text[..97])
114114+ }
115115+ }
116116+}
117117+118118+impl SourceFetcher for BlueskyFetcher {
119119+ fn sync(&self, storage: &dyn Storage) -> Result<()> {
120120+ let runtime =
121121+ tokio::runtime::Runtime::new().map_err(|e| PaiError::Fetch(format!("Failed to create runtime: {e}")))?;
122122+123123+ runtime.block_on(async {
124124+ let response = self.fetch_author_feed().await?;
125125+126126+ for feed_post in response.feed {
127127+ if !Self::is_original_post(&feed_post) {
128128+ continue;
129129+ }
130130+131131+ let post = feed_post.post;
132132+ let text = Self::extract_text(&post.record);
133133+134134+ let title = text.as_ref().map(|t| Self::create_title(t));
135135+ let url = Self::at_uri_to_url(&post.uri, &post.author.handle)?;
136136+137137+ let published_at = post
138138+ .record
139139+ .get("createdAt")
140140+ .and_then(|v| v.as_str())
141141+ .map(String::from)
142142+ .unwrap_or_else(|| Utc::now().to_rfc3339());
143143+144144+ let item = Item {
145145+ id: post.uri.clone(),
146146+ source_kind: SourceKind::Bluesky,
147147+ source_id: self.config.handle.clone(),
148148+ author: Some(post.author.handle.clone()),
149149+ title,
150150+ summary: text,
151151+ url,
152152+ content_html: None,
153153+ published_at,
154154+ created_at: Utc::now().to_rfc3339(),
155155+ };
156156+157157+ storage.insert_or_replace_item(&item)?;
158158+ }
159159+160160+ Ok(())
161161+ })
162162+ }
163163+}
164164+165165+#[cfg(test)]
166166+mod tests {
167167+ use super::*;
168168+169169+ #[test]
170170+ fn at_uri_to_url_valid() {
171171+ let uri = "at://did:plc:abc123/app.bsky.feed.post/xyz789";
172172+ let url = BlueskyFetcher::at_uri_to_url(uri, "user.bsky.social").unwrap();
173173+ assert_eq!(url, "https://bsky.app/profile/user.bsky.social/post/xyz789");
174174+ }
175175+176176+ #[test]
177177+ fn at_uri_to_url_invalid() {
178178+ let uri = "invalid-uri";
179179+ assert!(BlueskyFetcher::at_uri_to_url(uri, "user.bsky.social").is_err());
180180+ }
181181+182182+ #[test]
183183+ fn create_title_short_text() {
184184+ let text = "Short post";
185185+ assert_eq!(BlueskyFetcher::create_title(text), "Short post");
186186+ }
187187+188188+ #[test]
189189+ fn create_title_long_text() {
190190+ let text = "This is a very long post that exceeds one hundred characters and should be truncated with ellipsis at the end";
191191+ let title = BlueskyFetcher::create_title(text);
192192+ assert!(title.ends_with("..."));
193193+ assert_eq!(title.len(), 100);
194194+ }
195195+196196+ #[test]
197197+ fn extract_text_from_record() {
198198+ let record = serde_json::json!({
199199+ "text": "Hello world",
200200+ "createdAt": "2024-01-01T12:00:00Z"
201201+ });
202202+ let text = BlueskyFetcher::extract_text(&record).unwrap();
203203+ assert_eq!(text, "Hello world");
204204+ }
205205+206206+ #[test]
207207+ fn extract_text_missing() {
208208+ let record = serde_json::json!({
209209+ "createdAt": "2024-01-01T12:00:00Z"
210210+ });
211211+ assert!(BlueskyFetcher::extract_text(&record).is_none());
212212+ }
213213+214214+ #[test]
215215+ fn is_original_post_true() {
216216+ let feed_post = FeedViewPost {
217217+ post: PostView {
218218+ uri: "at://test".to_string(),
219219+ cid: "cid123".to_string(),
220220+ author: Author {
221221+ did: "did:plc:test".to_string(),
222222+ handle: "test.bsky.social".to_string(),
223223+ display_name: None,
224224+ },
225225+ record: serde_json::json!({}),
226226+ indexed_at: "2024-01-01T12:00:00Z".to_string(),
227227+ },
228228+ reason: None,
229229+ };
230230+ assert!(BlueskyFetcher::is_original_post(&feed_post));
231231+ }
232232+233233+ #[test]
234234+ fn is_original_post_false_repost() {
235235+ let feed_post = FeedViewPost {
236236+ post: PostView {
237237+ uri: "at://test".to_string(),
238238+ cid: "cid123".to_string(),
239239+ author: Author {
240240+ did: "did:plc:test".to_string(),
241241+ handle: "test.bsky.social".to_string(),
242242+ display_name: None,
243243+ },
244244+ record: serde_json::json!({}),
245245+ indexed_at: "2024-01-01T12:00:00Z".to_string(),
246246+ },
247247+ reason: Some(serde_json::json!({"$type": "app.bsky.feed.defs#reasonRepost"})),
248248+ };
249249+ assert!(!BlueskyFetcher::is_original_post(&feed_post));
250250+ }
251251+}
+133
core/src/fetchers/leaflet.rs
···11+use crate::{Item, LeafletConfig, PaiError, Result, SourceFetcher, SourceKind, Storage};
22+use chrono::Utc;
33+use feed_rs::parser;
44+55+/// Fetcher for Leaflet publications via RSS
66+///
77+/// Retrieves posts from Leaflet publications by parsing their RSS feeds.
88+/// Each Leaflet publication provides an RSS feed at {base_url}/rss.
99+pub struct LeafletFetcher {
1010+ config: LeafletConfig,
1111+ client: reqwest::Client,
1212+}
1313+1414+impl LeafletFetcher {
1515+ /// Creates a new Leaflet fetcher with the given configuration
1616+ pub fn new(config: LeafletConfig) -> Self {
1717+ Self { config, client: reqwest::Client::new() }
1818+ }
1919+2020+ /// Fetches and parses the RSS feed
2121+ async fn fetch_feed(&self) -> Result<feed_rs::model::Feed> {
2222+ let feed_url = format!("{}/rss", self.config.base_url.trim_end_matches('/'));
2323+ let response = self
2424+ .client
2525+ .get(&feed_url)
2626+ .send()
2727+ .await
2828+ .map_err(|e| PaiError::Fetch(format!("Failed to fetch Leaflet RSS feed: {e}")))?;
2929+3030+ let body = response
3131+ .text()
3232+ .await
3333+ .map_err(|e| PaiError::Fetch(format!("Failed to read response body: {e}")))?;
3434+3535+ parser::parse(body.as_bytes()).map_err(|e| PaiError::Parse(format!("Failed to parse RSS feed: {e}")))
3636+ }
3737+}
3838+3939+impl SourceFetcher for LeafletFetcher {
4040+ fn sync(&self, storage: &dyn Storage) -> Result<()> {
4141+ let runtime =
4242+ tokio::runtime::Runtime::new().map_err(|e| PaiError::Fetch(format!("Failed to create runtime: {e}")))?;
4343+4444+ runtime.block_on(async {
4545+ let feed = self.fetch_feed().await?;
4646+4747+ for entry in feed.entries {
4848+ let id = entry.id.clone();
4949+ let url = entry
5050+ .links
5151+ .first()
5252+ .map(|link| link.href.clone())
5353+ .unwrap_or_else(|| id.clone());
5454+5555+ let title = entry.title.as_ref().map(|t| t.content.clone());
5656+ let summary = entry.summary.as_ref().map(|s| s.content.clone());
5757+ let author = entry.authors.first().map(|a| a.name.clone());
5858+ let content_html = entry.content.and_then(|c| c.body);
5959+6060+ let published_at = entry
6161+ .published
6262+ .or(entry.updated)
6363+ .map(|dt| dt.to_rfc3339())
6464+ .unwrap_or_else(|| Utc::now().to_rfc3339());
6565+6666+ let item = Item {
6767+ id,
6868+ source_kind: SourceKind::Leaflet,
6969+ source_id: self.config.id.clone(),
7070+ author,
7171+ title,
7272+ summary,
7373+ url,
7474+ content_html,
7575+ published_at,
7676+ created_at: Utc::now().to_rfc3339(),
7777+ };
7878+7979+ storage.insert_or_replace_item(&item)?;
8080+ }
8181+8282+ Ok(())
8383+ })
8484+ }
8585+}
8686+8787+#[cfg(test)]
8888+mod tests {
8989+ use super::*;
9090+9191+ #[test]
9292+ fn parse_valid_rss() {
9393+ let rss = r#"<?xml version="1.0" encoding="UTF-8"?>
9494+<rss version="2.0">
9595+<channel>
9696+ <title>Test Leaflet</title>
9797+ <link>https://test.leaflet.pub</link>
9898+ <description>Test publication</description>
9999+ <item>
100100+ <title>Test Post</title>
101101+ <link>https://test.leaflet.pub/test-post</link>
102102+ <guid>test-guid</guid>
103103+ <pubDate>Mon, 01 Jan 2024 12:00:00 +0000</pubDate>
104104+ <description>Test summary</description>
105105+ </item>
106106+</channel>
107107+</rss>"#;
108108+109109+ let feed = parser::parse(rss.as_bytes()).unwrap();
110110+ assert_eq!(feed.entries.len(), 1);
111111+ assert_eq!(feed.entries[0].title.as_ref().unwrap().content, "Test Post");
112112+ }
113113+114114+ #[test]
115115+ fn parse_invalid_rss() {
116116+ let invalid_rss = "this is not valid XML";
117117+ let result = parser::parse(invalid_rss.as_bytes());
118118+ assert!(result.is_err());
119119+ }
120120+121121+ #[test]
122122+ fn parse_empty_rss() {
123123+ let rss = r#"<?xml version="1.0" encoding="UTF-8"?>
124124+<rss version="2.0">
125125+<channel>
126126+ <title>Empty Feed</title>
127127+</channel>
128128+</rss>"#;
129129+130130+ let feed = parser::parse(rss.as_bytes()).unwrap();
131131+ assert_eq!(feed.entries.len(), 0);
132132+ }
133133+}
+7
core/src/fetchers/mod.rs
···11+mod bluesky;
22+mod leaflet;
33+mod substack;
44+55+pub use bluesky::BlueskyFetcher;
66+pub use leaflet::LeafletFetcher;
77+pub use substack::SubstackFetcher;
+188
core/src/fetchers/substack.rs
···11+use crate::{Item, PaiError, Result, SourceFetcher, SourceKind, Storage, SubstackConfig};
22+use chrono::Utc;
33+use feed_rs::parser;
44+55+/// Fetcher for Substack RSS feeds
66+///
77+/// Retrieves posts from a Substack publication by parsing its RSS feed.
88+/// Maps RSS items to the standardized Item struct for storage.
99+pub struct SubstackFetcher {
1010+ config: SubstackConfig,
1111+ client: reqwest::Client,
1212+}
1313+1414+impl SubstackFetcher {
1515+ /// Creates a new Substack fetcher with the given configuration
1616+ pub fn new(config: SubstackConfig) -> Self {
1717+ Self { config, client: reqwest::Client::new() }
1818+ }
1919+2020+ /// Fetches and parses the RSS feed
2121+ async fn fetch_feed(&self) -> Result<feed_rs::model::Feed> {
2222+ let feed_url = format!("{}/feed", self.config.base_url);
2323+ let response = self
2424+ .client
2525+ .get(&feed_url)
2626+ .send()
2727+ .await
2828+ .map_err(|e| PaiError::Fetch(format!("Failed to fetch RSS feed: {e}")))?;
2929+3030+ let body = response
3131+ .text()
3232+ .await
3333+ .map_err(|e| PaiError::Fetch(format!("Failed to read response body: {e}")))?;
3434+3535+ parser::parse(body.as_bytes()).map_err(|e| PaiError::Parse(format!("Failed to parse RSS feed: {e}")))
3636+ }
3737+3838+ /// Extracts the source ID from the base URL (e.g., "patternmatched.substack.com")
3939+ fn extract_source_id(&self) -> String {
4040+ self.config
4141+ .base_url
4242+ .trim_start_matches("https://")
4343+ .trim_start_matches("http://")
4444+ .trim_end_matches('/')
4545+ .to_string()
4646+ }
4747+}
4848+4949+impl SourceFetcher for SubstackFetcher {
5050+ fn sync(&self, storage: &dyn Storage) -> Result<()> {
5151+ let runtime =
5252+ tokio::runtime::Runtime::new().map_err(|e| PaiError::Fetch(format!("Failed to create runtime: {e}")))?;
5353+5454+ runtime.block_on(async {
5555+ let feed = self.fetch_feed().await?;
5656+ let source_id = self.extract_source_id();
5757+5858+ for entry in feed.entries {
5959+ let id = entry.id.clone();
6060+ let url = entry
6161+ .links
6262+ .first()
6363+ .map(|link| link.href.clone())
6464+ .unwrap_or_else(|| id.clone());
6565+6666+ let title = entry.title.as_ref().map(|t| t.content.clone());
6767+ let summary = entry.summary.as_ref().map(|s| s.content.clone());
6868+ let author = entry.authors.first().map(|a| a.name.clone());
6969+ let content_html = entry.content.and_then(|c| c.body);
7070+7171+ let published_at = entry
7272+ .published
7373+ .or(entry.updated)
7474+ .map(|dt| dt.to_rfc3339())
7575+ .unwrap_or_else(|| Utc::now().to_rfc3339());
7676+7777+ let item = Item {
7878+ id,
7979+ source_kind: SourceKind::Substack,
8080+ source_id: source_id.clone(),
8181+ author,
8282+ title,
8383+ summary,
8484+ url,
8585+ content_html,
8686+ published_at,
8787+ created_at: Utc::now().to_rfc3339(),
8888+ };
8989+9090+ storage.insert_or_replace_item(&item)?;
9191+ }
9292+9393+ Ok(())
9494+ })
9595+ }
9696+}
9797+9898+#[cfg(test)]
9999+mod tests {
100100+ use super::*;
101101+ use crate::ListFilter;
102102+ use std::sync::{Arc, Mutex};
103103+104104+ #[derive(Clone)]
105105+ #[allow(dead_code)]
106106+ struct MockStorage {
107107+ items: Arc<Mutex<Vec<Item>>>,
108108+ }
109109+110110+ #[allow(dead_code)]
111111+ impl MockStorage {
112112+ fn new() -> Self {
113113+ Self { items: Arc::new(Mutex::new(Vec::new())) }
114114+ }
115115+116116+ fn get_items(&self) -> Vec<Item> {
117117+ self.items.lock().unwrap().clone()
118118+ }
119119+ }
120120+121121+ impl Storage for MockStorage {
122122+ fn insert_or_replace_item(&self, item: &Item) -> Result<()> {
123123+ self.items.lock().unwrap().push(item.clone());
124124+ Ok(())
125125+ }
126126+127127+ fn list_items(&self, _filter: &ListFilter) -> Result<Vec<Item>> {
128128+ Ok(self.items.lock().unwrap().clone())
129129+ }
130130+ }
131131+132132+ #[test]
133133+ fn extract_source_id_https() {
134134+ let config = SubstackConfig { enabled: true, base_url: "https://patternmatched.substack.com".to_string() };
135135+ let fetcher = SubstackFetcher::new(config);
136136+ assert_eq!(fetcher.extract_source_id(), "patternmatched.substack.com");
137137+ }
138138+139139+ #[test]
140140+ fn extract_source_id_http() {
141141+ let config = SubstackConfig { enabled: true, base_url: "http://test.substack.com/".to_string() };
142142+ let fetcher = SubstackFetcher::new(config);
143143+ assert_eq!(fetcher.extract_source_id(), "test.substack.com");
144144+ }
145145+146146+ #[test]
147147+ fn parse_valid_rss() {
148148+ let rss = r#"<?xml version="1.0" encoding="UTF-8"?>
149149+<rss version="2.0">
150150+<channel>
151151+ <title>Test Feed</title>
152152+ <link>https://test.substack.com</link>
153153+ <description>Test</description>
154154+ <item>
155155+ <title>Test Post</title>
156156+ <link>https://test.substack.com/p/test-post</link>
157157+ <guid>test-guid</guid>
158158+ <pubDate>Mon, 01 Jan 2024 12:00:00 +0000</pubDate>
159159+ <description>Test summary</description>
160160+ </item>
161161+</channel>
162162+</rss>"#;
163163+164164+ let feed = parser::parse(rss.as_bytes()).unwrap();
165165+ assert_eq!(feed.entries.len(), 1);
166166+ assert_eq!(feed.entries[0].title.as_ref().unwrap().content, "Test Post");
167167+ }
168168+169169+ #[test]
170170+ fn parse_invalid_rss() {
171171+ let invalid_rss = "this is not valid XML";
172172+ let result = parser::parse(invalid_rss.as_bytes());
173173+ assert!(result.is_err());
174174+ }
175175+176176+ #[test]
177177+ fn parse_empty_rss() {
178178+ let rss = r#"<?xml version="1.0" encoding="UTF-8"?>
179179+<rss version="2.0">
180180+<channel>
181181+ <title>Test Feed</title>
182182+</channel>
183183+</rss>"#;
184184+185185+ let feed = parser::parse(rss.as_bytes()).unwrap();
186186+ assert_eq!(feed.entries.len(), 0);
187187+ }
188188+}