Toot toooooooot (Bluesky-Mastodon cross-poster)

Compare changes

Choose any two refs to compare.

+1 -1
.gitignore
··· 1 1 .DS_Store 2 - config/*.csv 3 2 config/*.yml 3 + db/history.sqlite3*
+3
Gemfile
··· 8 8 gem 'net-http', '~> 0.2' 9 9 gem 'uri', '~> 0.13' 10 10 gem 'yaml', '~> 0.1' 11 + 12 + gem "activerecord", "~> 7.2" 13 + gem "sqlite3", "~> 2.7"
+49 -8
Gemfile.lock
··· 1 1 GEM 2 2 remote: https://rubygems.org/ 3 3 specs: 4 - base64 (0.2.0) 5 - didkit (0.2.3) 6 - io-console (0.7.2) 7 - json (2.10.2) 4 + activemodel (7.2.3) 5 + activesupport (= 7.2.3) 6 + activerecord (7.2.3) 7 + activemodel (= 7.2.3) 8 + activesupport (= 7.2.3) 9 + timeout (>= 0.4.0) 10 + activesupport (7.2.3) 11 + base64 12 + benchmark (>= 0.3) 13 + bigdecimal 14 + concurrent-ruby (~> 1.0, >= 1.3.1) 15 + connection_pool (>= 2.2.5) 16 + drb 17 + i18n (>= 1.6, < 2) 18 + logger (>= 1.4.2) 19 + minitest (>= 5.1) 20 + securerandom (>= 0.3) 21 + tzinfo (~> 2.0, >= 2.0.5) 22 + base64 (0.3.0) 23 + benchmark (0.5.0) 24 + bigdecimal (4.0.1) 25 + concurrent-ruby (1.3.6) 26 + connection_pool (3.0.2) 27 + didkit (0.3.1) 28 + drb (2.2.3) 29 + i18n (1.14.8) 30 + concurrent-ruby (~> 1.0) 31 + io-console (0.8.2) 32 + json (2.18.0) 33 + logger (1.7.0) 34 + mini_portile2 (2.8.9) 8 35 minisky (0.5.0) 9 36 base64 (~> 0.1) 10 - net-http (0.4.1) 11 - uri 12 - uri (0.13.2) 13 - yaml (0.3.0) 37 + minitest (6.0.1) 38 + prism (~> 1.5) 39 + net-http (0.9.1) 40 + uri (>= 0.11.1) 41 + prism (1.7.0) 42 + securerandom (0.4.1) 43 + sqlite3 (2.9.0) 44 + mini_portile2 (~> 2.8.0) 45 + sqlite3 (2.9.0-aarch64-linux-gnu) 46 + sqlite3 (2.9.0-arm64-darwin) 47 + sqlite3 (2.9.0-x86_64-linux-gnu) 48 + timeout (0.6.0) 49 + tzinfo (2.0.6) 50 + concurrent-ruby (~> 1.0) 51 + uri (0.13.3) 52 + yaml (0.4.0) 14 53 15 54 PLATFORMS 16 55 aarch64-linux ··· 19 58 x86_64-linux 20 59 21 60 DEPENDENCIES 61 + activerecord (~> 7.2) 22 62 didkit (~> 0.2) 23 63 io-console (~> 0.5) 24 64 json (~> 2.5) 25 65 minisky (~> 0.5) 26 66 net-http (~> 0.2) 67 + sqlite3 (~> 2.7) 27 68 uri (~> 0.13) 28 69 yaml (~> 0.1) 29 70
+9 -4
README.md
··· 20 20 21 21 ## Installation 22 22 23 - At the moment: 23 + To run this tool, you need some reasonably recent version of Ruby installed โ€“ although it's recommended to use a version that's still getting maintainance updates, i.e. currently 3.2+. A recent Ruby version is likely to be preinstalled on most Linux systems, or at least available through the OS's package manager, otherwise you can install one using tools such as [RVM](https://rvm.io), [asdf](https://asdf-vm.com), [ruby-install](https://github.com/postmodern/ruby-install) or [ruby-build](https://github.com/rbenv/ruby-build) (see more installation options on [ruby-lang.org](https://www.ruby-lang.org/en/downloads/)). 24 24 25 - git clone https://github.com/mackuba/tootify.git 25 + To install the app, run: 26 + 27 + git clone https://tangled.org/mackuba.eu/tootify 26 28 cd tootify 27 29 bundle install 28 30 ··· 55 57 56 58 * `bluesky.yml` โ€“ created when you log in, stores Bluesky user ID/password and access tokens 57 59 * `mastodon.yml` โ€“ created when you log in, stores Mastodon user ID/password and access tokens 58 - * `history.csv` โ€“ stores a mapping between Bluesky and Mastodon post IDs; used for reply references in threads 59 60 * `tootify.yml` - optional additional configuration 60 61 61 62 The config in `tootify.yml` currently supports one option: 62 63 63 64 - `extract_link_from_quotes: true` โ€“ if enabled, posts which are quotes of someone else's post which includes a link will be "collapsed" into a normal post that just includes that link directly without the quote (so the link card on Mastodon will show info about the link and not the quoted bsky.app post) 64 65 66 + There is also an SQLite database file that's automatically created in `db/history.sqlite3`. It stores a mapping between Bluesky and Mastodon post IDs, and is used to maintain reply references in threads. 67 + 65 68 66 69 ## Credits 67 70 68 - Copyright ยฉ 2025 Kuba Suder ([@mackuba.eu](https://bsky.app/profile/mackuba.eu)). 71 + Copyright ยฉ 2025 Kuba Suder ([@mackuba.eu](https://bsky.app/profile/did:plc:oio4hkxaop4ao4wz2pp3f4cr)). 69 72 70 73 The code is available under the terms of the [zlib license](https://choosealicense.com/licenses/zlib/) (permissive, similar to MIT). 74 + 75 + Bug reports and pull requests are welcome ๐Ÿ˜Ž
+1 -3
app/bluesky_account.rb
··· 17 17 exit 1 18 18 end 19 19 20 - pds = did.get_document.pds_endpoint.gsub('https://', '') 21 - 22 - @sky.host = pds 20 + @sky.host = did.document.pds_host 23 21 @sky.user.id = handle 24 22 @sky.user.pass = password 25 23
+16
app/database.rb
··· 1 + require 'active_record' 2 + 3 + module Database 4 + DB_FILE = File.expand_path(File.join(__dir__, '..', 'db', 'history.sqlite3')) 5 + MIGRATIONS_PATH = File.expand_path(File.join(__dir__, '..', 'db', 'migrate')) 6 + 7 + def self.init 8 + ActiveRecord::Base.establish_connection(adapter: 'sqlite3', database: DB_FILE) 9 + run_migrations 10 + end 11 + 12 + def self.run_migrations 13 + migration_context = ActiveRecord::MigrationContext.new(MIGRATIONS_PATH) 14 + migration_context.migrate 15 + end 16 + end
+23 -4
app/mastodon_account.rb
··· 2 2 require_relative 'mastodon_api' 3 3 4 4 class MastodonAccount 5 - APP_NAME = "tootify" 5 + APP_NAME = "Tootify" 6 6 CONFIG_FILE = File.expand_path(File.join(__dir__, '..', 'config', 'mastodon.yml')) 7 - OAUTH_SCOPES = 'read:accounts read:statuses write:media write:statuses' 7 + 8 + OAUTH_SCOPES = [ 9 + 'read:accounts', 10 + 'read:statuses', 11 + 'read:search', 12 + 'write:media', 13 + 'write:statuses' 14 + ].join(' ') 8 15 9 16 def initialize 10 17 @config = File.exist?(CONFIG_FILE) ? YAML.load(File.read(CONFIG_FILE)) : {} ··· 52 59 api.register_oauth_app(APP_NAME, OAUTH_SCOPES) 53 60 end 54 61 55 - def post_status(text, media_ids = nil, parent_id = nil) 62 + def instance_info 56 63 instance = @config['handle'].split('@').last 57 64 api = MastodonAPI.new(instance, @config['access_token']) 58 - api.post_status(text, media_ids, parent_id) 65 + api.instance_info 66 + end 67 + 68 + def post_status(text, media_ids: nil, parent_id: nil, quoted_status_id: nil) 69 + instance = @config['handle'].split('@').last 70 + api = MastodonAPI.new(instance, @config['access_token']) 71 + api.post_status(text, media_ids: media_ids, parent_id: parent_id, quoted_status_id: quoted_status_id) 59 72 end 60 73 61 74 def upload_media(data, filename, content_type, alt = nil) ··· 63 76 api = MastodonAPI.new(instance, @config['access_token']) 64 77 api.upload_media(data, filename, content_type, alt) 65 78 end 79 + 80 + def search_post_by_url(url) 81 + instance = @config['handle'].split('@').last 82 + api = MastodonAPI.new(instance, @config['access_token']) 83 + api.search_post_by_url(url) 84 + end 66 85 end
+16 -1
app/mastodon_api.rb
··· 74 74 get_json("/accounts/verify_credentials") 75 75 end 76 76 77 + def instance_info 78 + get_json("https://#{@host}/api/v2/instance") 79 + end 80 + 77 81 def lookup_account(username) 78 82 json = get_json("/accounts/lookup", { acct: username }) 79 83 raise UnexpectedResponseError.new unless json.is_a?(Hash) && json['id'].is_a?(String) ··· 84 88 get_json("/accounts/#{user_id}/statuses", params) 85 89 end 86 90 87 - def post_status(text, media_ids = nil, parent_id = nil) 91 + def post_status(text, media_ids: nil, parent_id: nil, quoted_status_id: nil) 88 92 params = { status: text } 89 93 params['media_ids[]'] = media_ids if media_ids 90 94 params['in_reply_to_id'] = parent_id if parent_id 95 + params['quoted_status_id'] = quoted_status_id if quoted_status_id 91 96 92 97 post_json("/statuses", params) 93 98 end ··· 127 132 128 133 def get_media(media_id) 129 134 get_json("/media/#{media_id}") 135 + end 136 + 137 + def search_post_by_url(url) 138 + json = get_json("https://#{@host}/api/v2/search", { 139 + q: url, 140 + type: 'statuses', 141 + resolve: true 142 + }) 143 + 144 + json['statuses'] && json['statuses'][0] 130 145 end 131 146 132 147 def get_json(path, params = {})
+10
app/post.rb
··· 1 + require 'active_record' 2 + 3 + class Post < ActiveRecord::Base 4 + validates_presence_of :mastodon_id, :bluesky_rkey 5 + 6 + validates_length_of :mastodon_id, maximum: 50 7 + validates_length_of :bluesky_rkey, is: 13 8 + 9 + validates_uniqueness_of :bluesky_rkey 10 + end
-23
app/post_history.rb
··· 1 - class PostHistory 2 - HISTORY_FILE = File.expand_path(File.join(__dir__, '..', 'config', 'history.csv')) 3 - 4 - def initialize 5 - if File.exist?(HISTORY_FILE) 6 - @id_map = File.read(HISTORY_FILE).lines.map { |l| l.strip.split(',') }.then { |pairs| Hash[pairs] } 7 - else 8 - @id_map = {} 9 - end 10 - end 11 - 12 - def [](bluesky_rkey) 13 - @id_map[bluesky_rkey] 14 - end 15 - 16 - def add(bluesky_rkey, mastodon_id) 17 - @id_map[bluesky_rkey] = mastodon_id 18 - 19 - File.open(HISTORY_FILE, 'a') do |f| 20 - f.puts("#{bluesky_rkey},#{mastodon_id}") 21 - end 22 - end 23 - end
+68 -18
app/tootify.rb
··· 2 2 require 'yaml' 3 3 4 4 require_relative 'bluesky_account' 5 + require_relative 'database' 5 6 require_relative 'mastodon_account' 6 - require_relative 'post_history' 7 + require_relative 'post' 7 8 8 9 class Tootify 9 10 CONFIG_FILE = File.expand_path(File.join(__dir__, '..', 'config', 'tootify.yml')) ··· 13 14 def initialize 14 15 @bluesky = BlueskyAccount.new 15 16 @mastodon = MastodonAccount.new 16 - @history = PostHistory.new 17 17 @config = load_config 18 18 @check_interval = 60 19 + 20 + Database.init 19 21 end 20 22 21 23 def load_config ··· 57 59 58 60 next unless repo == @bluesky.did && collection == 'app.bsky.feed.post' 59 61 62 + if post = Post.find_by(bluesky_rkey: rkey) 63 + log "Post #{rkey} was already cross-posted, skipping" 64 + @bluesky.delete_record_at(like_uri) 65 + next 66 + end 67 + 60 68 begin 61 69 record = @bluesky.fetch_record(repo, collection, rkey) 62 70 rescue Minisky::ClientErrorResponse => e 63 - puts "Record not found: #{post_uri}" 71 + log "Record not found: #{post_uri}" 64 72 @bluesky.delete_record_at(like_uri) 65 73 next 66 74 end ··· 70 78 prepo = parent_uri.split('/')[2] 71 79 72 80 if prepo != @bluesky.did 73 - puts "Skipping reply to someone else" 81 + log "Skipping reply to someone else" 74 82 @bluesky.delete_record_at(like_uri) 75 83 next 76 84 else ··· 86 94 87 95 if reply = record['reply'] 88 96 parent_uri = reply['parent']['uri'] 89 - prkey = parent_uri.split('/')[4] 97 + parent_rkey = parent_uri.split('/')[4] 90 98 91 - if parent_id = @history[prkey] 92 - mastodon_parent_id = parent_id 99 + if parent_post = Post.find_by(bluesky_rkey: parent_rkey) 100 + mastodon_parent_id = parent_post.mastodon_id 93 101 else 94 - puts "Skipping reply to a post that wasn't cross-posted" 102 + log "Skipping reply to a post that wasn't cross-posted" 95 103 @bluesky.delete_record_at(like_uri) 96 104 next 97 105 end 98 106 end 99 107 100 108 response = post_to_mastodon(record, mastodon_parent_id) 101 - p response 109 + log(response) 110 + 111 + Post.create!(bluesky_rkey: rkey, mastodon_id: response['id']) 102 112 103 - @history.add(rkey, response['id']) 104 113 @bluesky.delete_record_at(like_uri) 105 114 end 106 115 end ··· 113 122 end 114 123 115 124 def post_to_mastodon(record, mastodon_parent_id = nil) 116 - p record 125 + log(record) 117 126 118 127 text = expand_facets(record) 119 128 ··· 126 135 127 136 if collection == 'app.bsky.feed.post' 128 137 link_to_append = bsky_post_link(repo, rkey) 138 + instance_info = @mastodon.instance_info 129 139 130 - if @config['extract_link_from_quotes'] 140 + if instance_info.dig('api_versions', 'mastodon').to_i >= 7 131 141 quoted_record = fetch_record_by_at_uri(quote_uri) 132 142 133 - if link_from_quote = link_embed(quoted_record) 134 - link_to_append = link_from_quote 143 + # TODO: we need to wait for Bridgy to add support for quote_authorizations 144 + quoted_post_url = quoted_record['bridgyOriginalUrl'] #|| "https://bsky.brid.gy/convert/ap/#{quote_uri}" 145 + 146 + if quoted_post_url && (local_post = @mastodon.search_post_by_url(quoted_post_url)) 147 + quote_policy = local_post.dig('quote_approval', 'current_user') 148 + 149 + if quote_policy == 'automatic' || quote_policy == 'manual' 150 + quote_id = local_post['id'] 151 + end 135 152 end 136 153 end 137 154 138 - append_link(text, link_to_append) unless text.include?(link_to_append) 155 + if !quote_id && @config['extract_link_from_quotes'] 156 + quoted_record ||= fetch_record_by_at_uri(quote_uri) 157 + quote_link = link_embed(quoted_record) 158 + 159 + if quote_link.nil? 160 + text_links = links_from_facets(quoted_record) 161 + quote_link = text_links.first if text_links.length == 1 162 + end 163 + 164 + if quote_link 165 + link_to_append = quote_link 166 + end 167 + end 168 + 169 + append_link(text, link_to_append) unless quote_id || text.include?(link_to_append) 139 170 end 140 171 end 141 172 ··· 175 206 text += "\n\n" + tags.map { |t| '#' + t.gsub(' ', '') }.join(' ') 176 207 end 177 208 178 - @mastodon.post_status(text, media_ids, mastodon_parent_id) 209 + @mastodon.post_status(text, media_ids: media_ids, parent_id: mastodon_parent_id, quoted_status_id: quote_id) 179 210 end 180 211 181 212 def fetch_record_by_at_uri(quote_uri) 182 213 repo, collection, rkey = quote_uri.split('/')[2..4] 183 - pds = DID.new(repo).get_document.pds_endpoint 214 + pds = DID.new(repo).document.pds_host 184 215 sky = Minisky.new(pds, nil) 185 216 resp = sky.get_request('com.atproto.repo.getRecord', { repo: repo, collection: collection, rkey: rkey }) 186 217 resp['value'] ··· 206 237 bytes.pack('C*').force_encoding('UTF-8') 207 238 end 208 239 240 + def links_from_facets(record) 241 + links = [] 242 + 243 + if facets = record['facets'] 244 + facets.each do |f| 245 + if link = f['features'].detect { |ft| ft['$type'] == 'app.bsky.richtext.facet#link' } 246 + links << link['uri'] 247 + end 248 + end 249 + end 250 + 251 + links.reject { |x| x.start_with?('https://bsky.app/hashtag/') } 252 + end 253 + 209 254 def link_embed(record) 210 255 if embed = record['embed'] 211 256 case embed['$type'] ··· 267 312 end 268 313 269 314 def append_link(text, link) 270 - if link =~ /^https:\/\/bsky\.app\/profile\/.+\/post\/.+/ 315 + if link =~ %r{^https://bsky\.app/profile/.+/post/.+} 271 316 text << "\n" unless text.end_with?("\n") 272 317 text << "\n" 273 318 text << "RE: " + link ··· 279 324 280 325 def bsky_post_link(repo, rkey) 281 326 "https://bsky.app/profile/#{repo}/post/#{rkey}" 327 + end 328 + 329 + def log(obj) 330 + text = obj.is_a?(String) ? obj : obj.inspect 331 + puts "[#{Time.now}] #{text}" 282 332 end 283 333 end
+10
db/migrate/001_create_posts.rb
··· 1 + class CreatePosts < ActiveRecord::Migration[7.2] 2 + def change 3 + create_table :posts do |t| 4 + t.string :bluesky_rkey, null: false 5 + t.string :mastodon_id, null: false 6 + end 7 + 8 + add_index :posts, :bluesky_rkey, unique: true 9 + end 10 + end
+3 -1
tootify
··· 4 4 require_relative 'app/tootify' 5 5 6 6 def run(argv) 7 + $stdout.sync = true 8 + 7 9 options, args = argv.partition { |x| x.start_with?('-') } 8 10 9 11 app = Tootify.new ··· 37 39 38 40 if name =~ /\A[^@]+@[^@]+\z/ 39 41 app.login_to_mastodon(name) 40 - elsif name =~ /\A@[^@]+\z/ 42 + elsif name =~ /\A@?[^@]+\z/ 41 43 app.login_to_bluesky(name) 42 44 elsif name.nil? 43 45 print_help