Changeset 937

Show
Ignore:
Timestamp:
06/09/06 09:28:16
Author:
miyagawa
Message:

merge from trunk

Files:

Legend:

Unmodified
Added
Removed
Modified
Copied
Moved

  • branches/feature-server/plagger/AUTHORS

    r856 r937  
    2525Manabu Ishii 
    2626woremacx 
    27 Tatsuya Noda 
     27Tatsuya Noda (topia) 
    2828Motokazu Sekine (cheebow) 
     29Ilmari Vacklin (wolverian) 

  • branches/feature-server/plagger/Changes

    r856 r937  
    11The latest, HTML version of this document is always available at http://plagger.org/trac.cgi/wiki/PlaggerChangeLog 
    22 
     3== 0.7.2 (2006/06/02) == 
     4 
     5=== Core === 
     6 
     7 * Make Cookies instances Singleton (suggested by Yappo) 
     8 * Calls env_proxy on Plagger::UserAgent so that HTTP_PROXY env variable is respected (woremacx) 
     9 * Add w3m cookie files support 
     10 * Aggregator::Simple globally overrides LibXML parser with Liberal if possible 
     11 * Recommend newest XML::Liberal 0.11 so that liberal XML parsing would take the best effect  
     12 * $ua->mirror can take $req so that plugins can add custom HTTP headers like Referer. #208 
     13 * Change flv MIME value to video/x-flv 
     14 * Media RSS support in RSS parsers to extract enclosures and thumbnails. #141 
     15 * XML::Feed, XML::Atom, XML::LibXML and XML::RSS::LibXML are now core required modules. #176 
     16 * Support Hatena Fotolife and Apple photocast namespaces for thumbnail extraction.  
     17 * Filter::FetchEnclosure::Xango: Xango-based asynchronous POE downloader (dmaki) 
     18 
     19=== New Plugins === 
     20 
     21 * Filter::LivedoorKeywordUnlink: strip and normalize links to livedoor keywords (miyagawa) 
     22 * Subscription::File: subscribe to URLs in plain text file (wolverian) 
     23 * Subscription::FOAF: subscribe to friends' weblogs in FOAF:knows (wolverian) 
     24 * Subscription::Feed: subscribe to item links in RSS/Atom Feeds (miyagawa) 
     25 * Filter::FetchEnclosure::Xango: Xango-based asynchronous POE downloader (dmaki) 
     26 * Filter::FetchEnclosure::Wget: use wget command line tool for async download (miyagawa) 
     27 * Filter::FetchEnclosure::ParallelUA: use LWP::Parallel for async download (miyagawa) 
     28   
     29=== Plugins Updates === 
     30 
     31 * Filter::2chRSSContent: Updated regexp so that it works without milliseconds (miagawa) 
     32 * Filter::FeedBurnerPermalink: Work with Atom 1.0 feed to extract permalink 
     33 * Subscription::Bloglines: Try with mark_read=0, then fallback to loop mode if it failed, to avoid data loss 
     34 * CustomFeed::YouTube: updated regexp for thumbnail servers (fujiwara) 
     35 * Filter::EntryFullText: move the place of hook before date parser to hack date format 
     36 * Filter::BreakEntriesToFeeds: Don't change feed title. #220 
     37 * Filter::FetchEnclosure: Added fake_referer config to add Referer header.  
     38 * Publish::Gmail: attaches inline img enclosures as multipart/related and replace img src with Content-IDs. 
     39 * Publish::Gmail: attach_enclosures is now optional, defaults to off. 
     40 * Fliter::RSSLiberalDateTime: Fixed bug around minus timezones 
     41 * Filter::EntryFullText: Support icon extraction 
     42 * Filter::EntryFullText: Added NoNetwork option to URI::Fetch so that it's much faster 
     43 * Filter::EntryFullText: thinkit.co.jp and pasonatech.co.jp (otsune) 
     44 * Subscription::Planet: Added new Bloglines feed search  
     45 * Filter::*Permalink: removed in favor of TruePermalink 
     46  
    347== 0.7.1 (2006/05/24) == 
    448 
    …… 
    1155 
    1256 * Notify::UpdatePing: notify updates via XMLRPC pings (miyagawa) 
     57 * Notify::Tiarra: notify updates via Tiarra control socket (topia) 
    1358 * Publish::PalmDoc: publish updates to PalmDoc (cheebow) 
    1459 * Publish::OutlineText: publish updates as outline text (cheebow) 
    …… 
    1863 * Filter::Regexp: now you can use utf-8 regular expressions (woremacx) 
    1964 * Widget::Delicious: Support one_click_post to automatically post by clicking (s_nobu) 
    20  * Filter::FetchEnclosures: Now it's extensible using meta-plugins. Added youtube.pl as an example. Thanks to mizzy 
     65 * Filter::FindEnclosures: Now it's extensible using meta-plugins. Added youtube.pl as an example. Thanks to mizzy 
    2166 * CustomFeed::Simple: deduplicate links by URLs. Don't add links associated with images without alt (miyagawa) 
    2267 * Filter::TruePermalink: Added YouTube, MSN Mainichi 
    …… 
    304349 * Notify::SSTP: Notify updates to Sakura Script Transfer Protocol (secondlife) 
    305350 * Publish::Playlog: Use Atom Publishing Protocol to post iTunes playlog (mizzy) 
    306  * CustomFeed::iTunesRecentPlay: iTunes' recent playlist as a custom feed (mizzy) 
    307  * SmartFeed::All: All updates as one feed (miyagawa, cwest) 
    308  * Publish::CSV: publish updates as Comma Separated Value format (naoya) 
    309  * Filter::HatenaFormat: filter hatena format (naoya) 
    310  * Publish::MT: publish updates using MT XML-RPC API (naoya) 
    311  * Publish::Planet: Planet-Planet clone in Plagger (cwest) 
    312  * Publish::OPML: publish OPML files based on subscription (miyagawa) 
    313  * CustomFeed::FlickrSearch: Use Flickr API to create custom feeds (cwest) 
    314  * CustomFeed::AmazonAssociateReportJP: Fetch Amazon's associate report (naoya) 
    315  * Filter::ImageInfo: fetch information of $feed->image and $entry->icon (miyagawa) 
    316  * Subscription::HatenaGroup: fetch Hatena Group blogs as subscription (tokuhirom) 
    317  * Filter::BlogPet: Strip BlogPet's post (naoya) 
    318  * Filter::SpamAssassin: Use SpamAssassin to strip spam posts (charsbar) 
    319  * Filter::RSSLiberalDateTime: parses pubDate sring liberally (miyagawa) 
    320  
    321 === Plugins Updates === 
    322  
    323  * Aggregator::Xango: Added conditional GET support #93 (Daisuke) 
    324  * CustomFeed::Mixi: Support links to deleted entry #96 (miyagawa) 
    325  * Aggregator::Simple: Fixed error handling bug when URL is 404 #97 (miyagawa) 
    326  * CustomFeed::POP3: Create one feed per one mail (tokuhirom) 
    327  * Publish::Gmail: Fixed bad MAIL FROM: when used with Sendmail (Thanks to maru.gs) 
    328  * Subscription::Config: Support array of just URLs (miyagawa) 
    329  * Publish::PSP: template typo fix in permalink (miyagawa) 
    330  * Rule::Fresh: no datetime field means it's fresh (miyagawa) 
    331  * Subscription::Bloglines: Liberal parser support. Fallbacks to loop mode when it still finds error (miyagawa) 
    332  * Aggregator::Simple: uses XML::RSS::Liberal if it's installed (miyagawa) 
    333  * Publish::Gmail: resizes feed and entry images when it's too big (miyagawa) 
    334  * Filter::StripRssAd: Support Pheedo ads pattern (miyagawa) 
    335  * Rule::Fresh: Handle empty dates better (cwest) 
    336  * Aggregator::Simple: handle Atom 1.0 date fields (miyagawa, cwest) 
    337  * Widget::HatenaBookmarkUsersCount: Support '#' in the URL (otsune) 
    338  * CustomFeed::Frepa: Fixed to support Frepa's new HTML (miyagawa) 
    339  
    340 == 0.5.5 (2006/03/03) == 
    341  
    342 === Core === 
    343  
    344  * Added --version and shorter -c & -v to plagger script 
    345  * $entry->body_text now decode HTML entities correctly. #82 
    346  * New Plagger::Cache framework, accessible via $plugin->cache 
    347  * Added cookie_jar method to cache. $plugin->cache->cookie_jar 
    348  * config.yaml is now found using FindBin, rather than the current directory (Naoya Ito) 
    349  * New AUTHORS: Yoshiki Kurihara, Fumiaki Yoshimatsu, Masafumi Otsune, Takeshi Nagayama and fuba 
    350  
    351 === New Plugins === 
    352  
    353  * Filter::HatenaDiaryKeywordLink: Automatically link Hatena Diary Keywords in entry body (tokuhirom) 
    354  * Publish::Feed: Output Atom and RSS feeds using XML::Feed (clouder) 
    355  * Filter::ResolveRelativeLink: Fix relative links in entry body (miyagawa) 
    356  * CustomFeed::Yahoo360JP: Login to Yahoo! 360 Japan and fetch blogs and blasts (miyagawa) 
    357  * Filter::BulkfeedsTerms: Use Bulkfeeds API to fetch specific terms in entry (miyagawa) 
    358  * Filter::2chNewsokuTitle: Add prefix and postfix to entry title ala 2ch.net Newsoku style (miyagawa) 
    359  * Notify::Eject: eject your CD drive to notify feed updates! :-) (Yappo, fumiakiy, otsune) 
    360  * Publish::Pipe: use UNIX pipe to notify the updates to commands (e.g: lpr, /usr/bin/say) (youpy) 
    361  * Notify::Campfire: Notify feed updates to 37 signals' Campfire chat service (nagayama) 
    362  * Filter::Regexp: Update entry body by a regular expression in config (miyagawa) 
    363  * Publish::HatenaBookmark: post updates to Hatena Bookmark using Atom Protocol (fuba) 
    364  * CustomFeed::POP3: Fetch email using POP3 protocl and creates feed off of it (tokuhirom) 
    365  * Filter::RSSTimeZoneString: Fix RSS 2.0 bad timezone string in RFC 822 date format (miyagawa) 
    366  * Filter::FloatingDateTime: Fix floating datetime (dc:date and such) to your local timezone (kazeburo) 
    367  
    368 === Plugins Updates === 
    369   
    370  * Publish::Delicious: fix UTF-8 escape bug. 
    371  * Subscription::Bloglines: Added 'fetch_meta' option to fetch folder structure and feed URL 
    372  * Subscription::Bloglines: Fix annoying UTF-8 bad sequences by upgrading to WebService::Bloglines 0.10 
    373  * Filter::TTP: Added 'text_only' option to filter text element only in HTML. #76 
    374  * Fitler::StripRSSAd: Now it works with Bloglines subscription. Added Google AdSense pattern. 
    375  * Publish::IRC, Publish::MSAgent and Publish::Growl are now renamed to Notify::* 
    376  * Publish::Gmail: fixed typo in "Permalink" footer 
    377  * CustomFeed::Mixi, CustomFeed::Frepa now uses Cache framework to store persistent cookies  
    378  * CustomFeed::Frepa: support PNG buddyicon 
    379  * Publish::Gmail: do POP before SMTP authentication only once per whole publish 
    380  * Subscription::OPML and Aggregator::Simple now use URI::Fetch and Cache framework for conditional GET 
    381  
    382 == 0.5.4 (2006/02/27) == 
    383  
    384 === Core === 
    385  
    386  * Makefile.PL has a much better dependencies definition #59 (Thanks to Daisuke Maki) 
    387  * Fixed multiple categories handling bug in Aggregator::Simple (Thanks to koyachi) 
    388  * Added body_text, title_text convinience methods to Feed and Entry 
    389  * Now uses FindBin module to find templates directory in a better way (youpy) 
    390  * New authors: Daisuke Maki, Tokuhiro Matsuno and Tsutomu Koyachi 
    391  
    392 === New Plugins === 
    393  
    394  * Aggregator::Xango - POE based high-speed parallel crawling (daisuke) 
    395  * Publish::PDF - Create PDF files based on feeds (miyagawa) 
    396  * Publish::Speech - Make the feeds as an audio using Text-to-Speech (miyagawa) 
    397  * Publish::MSAgent - Let Microsoft Agent speak feeds content (miyagawa) 
    398  * Publish::Growl - Send feed notifications to Mac OSX Growl (kazeburo) 
    399  * Filter::tDiaryComment - strip comments from tDiary RSS (tokuhirom) 
    400  * Filter::TTP - replace ttp: with http: (tokuhirom) 
    401  * Filter::DeliciousFeedTags - split del.icio.us "foo bar baz" tags into array (miyagawa) 
    402  * Publish::Delicious - post entries to del.icio.us automatically (koyachi) 
    403  
    404 === Plugins Updates === 
    405  
    406  * Publish::Spotlight: Now it fallbacks to osascript when Mac::Glue is not installed 
    407  * Publish::MTWidget: Added an option to rebuild the blog templates after updating widgets 
    408  * Publish::Gmail: Added an option to support POP3 before SMTP (tokuhirom) 
    409  
    410 == 0.5.3 (2006/02/25) == 
    411  
    412 === Core === 
    413  
    414  * Added aggregator.finalize, publish.entry and publish.init hook 
    415  * Added Plagger::Rule::Fresh to match with 'fresh' entries (thanks to youpy) 
    416  * Fixed bug with SmartFeed creating duplicated entries #61 
    417  * Support log:level global configuration flag to set min loglevel 
    418  
    419 === Plugins === 
    420  
    421  * Added Publish::Spotlight to create WebBookmark searchable with Spotlight (youpy) 
    422  * Added Search::Namazu to create HTML files searchable with Namazu (miyagawa) 
    423  * Added Search::Rast to index entries with Rast (Yappo) 
    424  * Added Publish::MTWidget to publish Movable Type Sidebar Manager widget (miyagawa) 
    425  * Added an option 'show_icon: 1', to show users pic in Frepa and Mixi custom feeds 
    426  * Fixed Subscription::Mailman to handle subject prefixes without index 
    427  * Publish::IRC now has 'announce: action' option to use CTCP ACTION rather than NOTICE 
    428  * Filter::StripRSSAd now strips Rakuten Hiroba (plaza.rakuten.co.jp) ad pattern 
    429  * Fixed CustomFeed::Frepa to complete the trimmed title under fetch_body mode 
    430  * Fixed typo in Subscription::HatenaRSS 
    431  
    432 == 0.5.2 (2006/02/22) == 
    433  
    434  * Added Subscription::HatenaRSS plugin to fetch subscription from Hatena RSS (http://r.hatena.ne.jp/) 
    435  * Added Subscription::Odeo plugin to fetch subscription from Odeo (http://www.odeo.com/): #43 
    436  * Added Publish::IRC plugin to notify updates to IRC channel: #46 (Masayoshi Sekimura) 
    437  * Fixed critical bug that Rule::Expression is broken: #51 
    438  * Fixed Mailman CustomFeed bug around English localization: #49 
    439  * Added Filter::RSSStripAd plugin to strip ads of Google AdSense or RssAd.jp: #33 
    440  
    441 == 0.5.1 (2006/02/17) == 
    442   
    443  * Updated package name from "plagger" to "Plagger" 
    444  * Fixed MANIFEST problem that doesn't contain "config.yaml.sample" 
    445  
    446 == 0.5.0 (2006/02/17) == 
    447  
    448  * First release 
     351 * CustomFeed::iTunesRecentPlay: iTunes' recent playlist 

  • branches/feature-server/plagger/MANIFEST

    r856 r937  
    1010assets/plugins/Filter-EntryFullText/bbc.yaml 
    1111assets/plugins/Filter-EntryFullText/blog_goo_ne_jp.yaml 
     12assets/plugins/Filter-EntryFullText/blog_pasonatech_co_jp.yaml 
    1213assets/plugins/Filter-EntryFullText/blog_tech.rikunabi_next.yaml 
    1314assets/plugins/Filter-EntryFullText/business-i.yaml 
    …… 
    3031assets/plugins/Filter-EntryFullText/itmedia.yaml 
    3132assets/plugins/Filter-EntryFullText/itpro_nikkeibp.yaml 
    32 assets/plugins/Filter-EntryFullText/japan_linux_com.yaml 
    3333assets/plugins/Filter-EntryFullText/japan_zdnet_com.yaml 
    3434assets/plugins/Filter-EntryFullText/kyodo.yaml 
    …… 
    4646assets/plugins/Filter-EntryFullText/nytimes.yaml 
    4747assets/plugins/Filter-EntryFullText/okinawatimes_day.yaml 
     48assets/plugins/Filter-EntryFullText/opentechpress_jp.yaml 
    4849assets/plugins/Filter-EntryFullText/osaka_nikkansports.yaml 
    4950assets/plugins/Filter-EntryFullText/physorg.yaml 
    …… 
    6465assets/plugins/Filter-EntryFullText/theinquirer.yaml 
    6566assets/plugins/Filter-EntryFullText/theregister.yaml 
     67assets/plugins/Filter-EntryFullText/thinkit_co_jp.yaml 
    6668assets/plugins/Filter-EntryFullText/ti-da_net.yaml 
    6769assets/plugins/Filter-EntryFullText/usatoday.yaml 
    …… 
    9193assets/plugins/Filter-TruePermalink/namaan.yaml 
    9294assets/plugins/Filter-TruePermalink/rd_yahoo.yaml 
    93 assets/plugins/Filter-TruePermalink/reddit.yaml 
    9495assets/plugins/Filter-TruePermalink/redirectors.yaml 
    9596assets/plugins/Filter-TruePermalink/refrss.yaml 
    …… 
    183184lib/Plagger/Plugin/Filter/2chNewsokuTitle.pm 
    184185lib/Plagger/Plugin/Filter/2chRSSContent.pm 
    185 lib/Plagger/Plugin/Filter/2chRSSPermalink.pm 
    186186lib/Plagger/Plugin/Filter/AtomLinkRelated.pm 
    187187lib/Plagger/Plugin/Filter/Base.pm 
    …… 
    198198lib/Plagger/Plugin/Filter/FeedFlareStripper.pm 
    199199lib/Plagger/Plugin/Filter/FetchEnclosure.pm 
     200lib/Plagger/Plugin/Filter/FetchEnclosure/ParallelUA.pm 
     201lib/Plagger/Plugin/Filter/FetchEnclosure/Wget.pm 
     202lib/Plagger/Plugin/Filter/FetchEnclosure/Xango.pm 
    200203lib/Plagger/Plugin/Filter/FindEnclosures.pm 
    201204lib/Plagger/Plugin/Filter/FloatingDateTime.pm 
    …… 
    208211lib/Plagger/Plugin/Filter/HEADEnclosureMetadata.pm 
    209212lib/Plagger/Plugin/Filter/ImageInfo.pm 
     213lib/Plagger/Plugin/Filter/LivedoorKeywordUnlink.pm 
    210214lib/Plagger/Plugin/Filter/Markdown.pm 
    211 lib/Plagger/Plugin/Filter/NamaanPermalink.pm 
    212215lib/Plagger/Plugin/Filter/Pipe.pm 
    213216lib/Plagger/Plugin/Filter/POPFile.pm 
    …… 
    229232lib/Plagger/Plugin/Filter/TTP.pm 
    230233lib/Plagger/Plugin/Filter/URLBL.pm 
    231 lib/Plagger/Plugin/Filter/YahooBlogSearchPermalink.pm 
    232234lib/Plagger/Plugin/Notify/Campfire.pm 
    233235lib/Plagger/Plugin/Notify/Eject.pm 
    …… 
    277279lib/Plagger/Plugin/Subscription/Config.pm 
    278280lib/Plagger/Plugin/Subscription/DBI.pm 
     281lib/Plagger/Plugin/Subscription/Feed.pm 
     282lib/Plagger/Plugin/Subscription/File.pm 
     283lib/Plagger/Plugin/Subscription/FOAF.pm 
    279284lib/Plagger/Plugin/Subscription/HatenaGroup.pm 
    280285lib/Plagger/Plugin/Subscription/HatenaRSS.pm 
    …… 
    316321plagger 
    317322t/00_compile.t 
     323t/core/cookies.t 
     324t/core/cookies.txt 
     325t/core/googlevideo.xml 
     326t/core/hatenafotolife.rdf 
     327t/core/hatenafotolife.t 
     328t/core/monkey.rss 
     329t/core/mrss.t 
     330t/core/photocast.rss 
     331t/core/photocast.t 

  • branches/feature-server/plagger/Makefile.PL

    r856 r937  
    2525requires('MIME::Types', 1.16); 
    2626 
     27requires('XML::Feed', 0.08); 
     28requires('XML::LibXML'); 
     29requires('XML::Atom', 0.19); 
     30requires('XML::RSS::LibXML', 0.20), 
     31 
    2732build_requires(Test::More => 0.42); 
    2833 
    …… 
    4146        recommends('HTTP::Cookies::Mozilla'), 
    4247    ], 
     48    'Liberal XML parsing' => [ 
     49        -default => 1, 
     50        recommends('XML::Liberal', 0.11), 
     51        recommends('XML::RSS::Liberal'), 
     52    ], 
    4353); 
    4454 
    …… 
    4757        -default => 1, 
    4858        recommends('WebService::Bloglines', 0.11), 
    49         recommends('XML::Liberal', 0.09), 
    5059    ], 
    5160    'Subscription::OPML' => [ 
    5261        -default => 1, 
    5362        recommends('XML::OPML'), 
    54     ], 
    55     'Aggregator::Simple' => [ 
    56         -default => 1, 
    57         recommends('XML::Feed', 0.08), 
    58         recommends('XML::Atom'), 
    59         recommends('XML::RSS::LibXML', 0.20), 
    60         recommends('XML::RSS::Liberal'), 
    6163    ], 
    6264    'Publish::Gmail' => [ 
    …… 
    6466        recommends('Mail::Address'), 
    6567        recommends('MIME::Lite'), 
     68        recommends('Net::SMTP::TLS'), 
    6669    ], 
    6770    'Aggregator::Xango' => [ 
    …… 
    6972        recommends('POE'), 
    7073        recommends('Xango', 1.04), # protect ourselves from my stupidity 
    71         recommends('XML::Feed', 0.08), 
    7274        recommends('Cache::FastMmap'), 
    7375        recommends('POE::Component::Client::DNS'), 
    …… 
    111113        recommends('XMLRPC::Lite'), 
    112114    ], 
    113     'Publish::Feed' => [ 
    114         -default => 1, 
    115         recommends('XML::Feed', 0.08), 
    116         recommends('XML::RSS::LibXML', 0.19), 
    117     ], 
    118115    'Filter::ResolveRelativeLink' => [ 
    119116        -default => 0, 
    120117        recommends('HTML::ResolveLink', 0.02), 
    121     ], 
    122     'Filter::BulkfeedsTerms' => [ 
    123         -default => 0, 
    124         recommends('XML::LibXML'), 
    125     ], 
    126     'Publish::HatenaBookmark' => [ 
    127         -default => 0, 
    128         recommends('XML::Atom'), 
    129     ], 
    130     'Publish::Playlog' => [ 
    131         -default => 0, 
    132         recommends('XML::Atom', 0.13), 
    133118    ], 
    134119    'Publish::Planet' => [ 
    …… 
    152137    'CustomFeed::FlickrSearch' => [ 
    153138        -default => 0, 
    154         recommends('XML::LibXML'), 
    155139        recommends('Flickr::API'), 
    156140        recommends('DateTime::Format::Epoch'), 
    …… 
    233217        -default => 0, 
    234218        recommends('Palm::PalmDoc'), 
     219    ], 
     220    'Subscription::FOAF' => [ 
     221        -default => 0, 
     222        recommends('XML::FOAF'), 
    235223    ], 
    236224); 

  • branches/feature-server/plagger/assets/plugins/Filter-EntryFullText/itpro_nikkeibp.yaml

    r616 r937  
    11# nikkeibp ITpro 
    22author: Manabu Ishii 
    3 handle: http://itpro\.nikkeibp\.co\.jp/ 
     3handle: http://itpro\.nikkeibp\.co\.jp/(?!article/Watcher) 
    44extract: <H2>(.*?)</H2>.*?<!-- start .*? -->(.*?)<!-- written --> 
    55extract_capture: title body 

  • branches/feature-server/plagger/assets/plugins/Filter-EntryFullText/sixapart.pl

    r539 r937  
    11sub handle { 
    22    my($self, $args) = @_; 
    3     $args->{content} =~ m!<meta name="generator" content="(?:http://www\.typepad\.com/|Movable Type.*?)" />!; 
     3    return 1 if $args->{content} =~ m!<html[^>]+id="sixapart-standard"!; 
     4    return 1 if $args->{content} =~ m!<meta name="generator" content="(?:http://www\.typepad\.com/|Movable Type.*?)" />!; 
     5    return; 
    46} 
    57 

  • branches/feature-server/plagger/assets/plugins/Filter-FindEnclosures/youtube.pl

    r856 r937  
    77 
    88sub find { 
    9     my ($self, $content) = @_; 
     9    my ($self, $args) = @_; 
    1010 
    11     if ($content =~ /video_id=([^&]+)&l=\d+&t=([^&]+)/gms){ 
     11    if ($args->{content} =~ /video_id=([^&]+)&l=\d+&t=([^&]+)/gms){ 
    1212        my $enclosure = Plagger::Enclosure->new; 
    1313        $enclosure->url("http://youtube.com/get_video?video_id=$1&t=$2"); 

  • branches/feature-server/plagger/assets/plugins/Filter-TruePermalink/reddit.yaml

    r669 r937  
    22match: http://(?:\w+\.)?reddit\.com/goto\? 
    33rewrite: | 
    4   if ($args->{entry}->body =~ m!^\s*<a href="[^\"]+".*?>\[link\]</a>!) { 
     4  if ($args->{entry}->body =~ m!^\s*<a href="([^\"]+)".*?>\[link\]</a>!) { 
    55      $_ = $1; 
    66      return 1; 

  • branches/feature-server/plagger/assets/plugins/Filter-TruePermalink/redirectors.yaml

    r856 r937  
    11- http://www\.pheedo\.jp/click.phdo\?i=\w+ 
    22- http://feeds\.feedburner\.(com|jp)/[\w/]+\?m=\d+ 
     3- http://feeds\.feedburner\.(com|jp)/~r/ 
    34- http://tinyurl\.com/\w+ 
    45- http://xrl\.us/\w+ 
    56- http://qurl\.net/\w+ 
     7- http://(?:\w+\.)?reddit\.com/goto\? 

  • branches/feature-server/plagger/assets/plugins/Publish-Planet/default/template/index.tt

    r456 r937  
    55  <head> 
    66    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> 
     7    <meta name="generator" contnet="Plagger [% context.VERSION %]" /> 
    78    <link rel="alternate" type="application/atom+xml" title="Atom: [% feed.title %]" href="smartfeed_all.atom" /> 
    89    <link rel="alternate" type="application/rss+xml" title="RSS: [% feed.title %]" href="smartfeed_all.rss" /> 

  • branches/feature-server/plagger/lib/Plagger.pm

    r856 r937  
    11package Plagger; 
    22use strict; 
    3 our $VERSION = '0.7.1'; 
     3our $VERSION = '0.7.2'; 
    44 
    55use 5.8.1; 

  • branches/feature-server/plagger/lib/Plagger/Entry.pm

    r856 r937  
    9494} 
    9595 
     96sub digest { 
     97    my $self = shift; 
     98    Digest::MD5::md5_hex($self->title . ($self->body || '')); 
     99} 
     100 
    961011; 
    97102 

  • branches/feature-server/plagger/lib/Plagger/Plugin/Aggregator/Simple.pm

    r856 r937  
    1515 
    1616eval { require XML::Liberal }; 
    17 if (!$@ && XML::Liberal->can('globally_override')) { 
     17if (!$@ && $XML::Liberal::VERSION >= 0.10) { 
    1818    XML::Liberal->globally_override('LibXML'); 
    1919} 
    …… 
    160160        } 
    161161 
     162        # TODO: move MediaRSS, Hatena, iTunes and those specific parser to be subclassed 
     163 
    162164        # Media RSS 
    163165        my $media_ns = "http://search.yahoo.com/mrss"; 
    …… 
    179181                height => $thumbnail->{height}, 
    180182            }); 
     183        } 
     184 
     185        # Hatena Image extensions 
     186        my $hatena = $e->{entry}->{"http://www.hatena.ne.jp/info/xmlns#"} || {}; 
     187        if ($hatena->{imageurl}) { 
     188            my $enclosure = Plagger::Enclosure->new; 
     189            $enclosure->url($hatena->{imageurl}); 
     190            $enclosure->auto_set_type; 
     191            $entry->add_enclosure($enclosure); 
     192        } 
     193 
     194        if ($hatena->{imageurlsmall}) { 
     195            $entry->icon({ url   => $hatena->{imageurlsmall} }); 
     196        } 
     197 
     198        # Apple photocast feed 
     199        my $apple = $e->{entry}->{"http://www.apple.com/ilife/wallpapers"} || {}; 
     200        if ($apple->{image}) { 
     201            my $enclosure = Plagger::Enclosure->new; 
     202            $enclosure->url( URI->new($apple->{image}) ); 
     203            $enclosure->auto_set_type; 
     204            $entry->add_enclosure($enclosure); 
     205        } 
     206        if ($apple->{thumbnail}) { 
     207            $entry->icon({ url => $apple->{thumbnail} }); 
    181208        } 
    182209 

  • branches/feature-server/plagger/lib/Plagger/Plugin/CustomFeed/FlickrSearch.pm

    r430 r937  
    77use XML::LibXML; 
    88use DateTime::Format::Epoch; 
     9use Plagger::Enclosure; 
    910 
    1011sub register { 
    …… 
    3031    my $feed = Plagger::Feed->new; 
    3132    $feed->type('flickr.search'); 
    32     $feed->title("Flickr Search"); 
    33     $feed->id('flickr:search'); 
     33    $feed->title("Flickr Search"); # xxx 
     34    $feed->id('flickr:search'); # xxx 
    3435 
    3536    my $flickr = Flickr::API->new({key => $self->conf->{api_key}}); 
    36     my $search = $flickr->execute_method('flickr.photos.search' => $self->conf); 
     37    my $method = $self->conf->{method} || 'flickr.photos.search'; 
     38 
     39    my $params = $self->conf->{params} || {}; 
     40    $params->{per_page} ||= 20; 
     41 
     42    $context->log(info => "calling $method on Flickr API"); 
     43    my $search = $self->call_method( 
     44        $flickr, 
     45        $method, 
     46        $params, 
     47        60 * 60, 
     48    ); 
    3749 
    3850    my $parser = XML::LibXML->new; 
    3951 
    40     $context->error("flickr.photos.search failed: $search->{error_text}") 
     52    $context->error("$method failed: $search->{error_text}") 
    4153      unless $search->{success}; 
    4254    my $search_doc = $parser->parse_string($search->{_content}); 
    …… 
    5264sub _create_entry { 
    5365    my ($self, $context, $flickr, $parser, $search_photo) = @_; 
    54     my $sizes = $flickr->execute_method('flickr.photos.getSizes', { 
    55         photo_id => $search_photo->findvalue('@id'), 
    56     }); 
    57     next unless $sizes->{success}; 
    58      
    59     my $sizes_doc = $parser->parse_string($sizes->{_content}); 
    60     my $size      = $self->conf->{size} || 'Square'; 
    61     my $image_src = 
    62       $sizes_doc->findvalue(qq[/rsp/sizes/size[\@label='$size']/\@url]); 
    6366 
    64     my $info = $flickr->execute_method('flickr.photos.getInfo', { 
    65         photo_id => $search_photo->findvalue('@id'), 
    66     }); 
     67    my $photo_id  = $search_photo->findvalue('@id'); 
     68    my $server_id = $search_photo->findvalue('@server'); 
     69    my $secret    = $search_photo->findvalue('@secret'); 
     70 
     71    my $size      = $self->conf->{size} || 'm'; 
     72    my $thumb_src = sprintf "http://static.flickr.com/%s/%s_%s_t.jpg", 
     73        $server_id, $photo_id, $secret; 
     74 
     75    $context->log(info => "calling flickr.photos.getInfo on $photo_id"); 
     76    my $info = $self->call_method( 
     77        $flickr, 
     78        'flickr.photos.getInfo', 
     79        { photo_id => $photo_id }, 
     80        60 * 60, 
     81    ); 
    6782    next unless $info->{success}; 
    68      
     83 
    6984    my $info_doc = $parser->parse_string($info->{_content}); 
    7085    my $link     = $info_doc->findvalue(q[/rsp/photo/urls/url[@type='photopage']]); 
    71     my $author   = $info_doc->findvalue(q[/rsp/photo/owner/@realname]); 
    72        $author   = $info_doc->findvalue(q[/rsp/photo/owner/@username]) unless $author; 
     86    my $author   = $info_doc->findvalue(q[/rsp/photo/owner/@realname]) 
     87                || $info_doc->findvalue(q[/rsp/photo/owner/@username]); 
    7388    my $title    = $info_doc->findvalue(q[/rsp/photo/title]); 
    7489    my $date     = $info_doc->findvalue(q[/rsp/photo/dates/@posted]); 
     90    my $format   = $info_doc->findvalue(q[/rsp/photo/@originalformat]) || 'jpg'; 
     91    my $desc     = $info_doc->findvalue(q[/rsp/photo/description]); 
     92    my @tags     = map $_->textContent, $info_doc->findnodes('/rsp/photo/tags/tag'); 
    7593 
    76     my $description = $context->templatize($self, 'entry-description.tt', { 
    77         image_src => $image_src, 
    78         title     => $title, 
    79         link      => $link, 
    80     }); 
     94    my $original = sprintf "http://static.flickr.com/%s/%s_%s_o.%s", 
     95        $server_id, $photo_id, $secret, $format; 
    8196    my $epoch = DateTime->from_epoch(epoch => 0, time_zone => '+0000'); 
    8297 
    …… 
    8499    $entry->title($title); 
    85100    $entry->link($link); 
    86     $entry->body($description); 
     101    $entry->author($author); 
     102    $entry->body($desc); 
    87103    $entry->date(Plagger::Date->parse('Epoch::Unix', $date)); 
     104    $entry->add_tag($_) for @tags; 
     105    $entry->icon({ url => $thumb_src }); 
     106 
     107    my $enclosure = Plagger::Enclosure->new; 
     108    $enclosure->url($original); 
     109    $enclosure->auto_set_type; 
     110    $entry->add_enclosure($enclosure); 
    88111 
    89112    return $entry; 
    90113} 
    91114 
     115sub call_method { 
     116    my($self, $flickr, $method, $param, $cache) = @_; 
     117 
     118    my $cache_key = "$method:" . join("|", map "$_=$param->{$_}", sort keys %$param); 
     119    $self->cache->get_callback( 
     120        $cache_key, 
     121        sub { $flickr->execute_method($method, $param) }, 
     122        $cache, 
     123    ); 
     124} 
     125 
    921261; 
     127 
     128__END__ 
     129 
     130=head1 NAME 
     131 
     132Plagger::Plugin::CustomFeed::FlickrSearch - Flickr API as Custom Feed 
     133 
     134=head1 SYNOPSIS 
     135 
     136  - module: CustomFeed::FlickrSearch 
     137    config: 
     138     api_key: YOUR-FLICKR-APIKEY 
     139     method: flickr.photos.search 
     140     params: 
     141       tags: plagger 
     142 
     143=head1 AUTHOR 
     144 
     145Casey West 
     146 
     147Tatsuhiko Miyagawa 
     148 
     149=head1 SEE ALSO 
     150 
     151L<Plagger>, L<http://www.flickr.com/>, L<Flickr::API> 
     152 
     153=cut 

  • branches/feature-server/plagger/lib/Plagger/Plugin/Filter/2chRSSContent.pm

    r856 r937  
    1717 
    1818    my $body = $args->{entry}->body; 
    19     if ($body =~ s!^([^:]*):(\d{4}/\d\d/\d\d)\(.*?\) (\d\d:\d\d:\d\d)(?:\.\d\d)? (ID:\S+)  ?!!) { 
     19    if ($body =~ s!^([^:]*):(\d{4}/\d\d/\d\d)\(.*?\) (\d\d:\d\d:\d\d)(?:\.\d\d)? (ID:\S+)?  ?!!) { 
    2020        my($from, $day, $time, $id) = ($1, $2, $3, $4); 
    2121 

  • branches/feature-server/plagger/lib/Plagger/Plugin/Filter/EntryFullText.pm

    r856 r937  
    9999 
    100100    my $handler = first { $_->handle_force($args) } @{ $self->{plugins} }; 
    101     if ( !$handler && $args->{entry}->body && $args->{entry}->body =~ /<\w+>/ ) { 
     101    if ( !$handler && $args->{entry}->body && $args->{entry}->body =~ /<\w+>/ && !$self->conf->{force_upgrade} ) { 
    102102        $self->log(debug => $args->{entry}->link . " already contains body. Skipped"); 
    103103        return; 
    …… 
    111111    # NoNetwork: don't connect for 3 hours 
    112112    my $res = $self->{ua}->fetch( $args->{entry}->permalink, $self, { NoNetwork => 60 * 60 * 3 } ); 
    113     return if $res->status != URI::Fetch::URI_OK && $res->is_error; 
     113    return if !$res->status && $res->is_error; 
    114114 
    115115    $args->{content} = decode_content($res); 
    …… 
    281281Defaults to 0. 
    282282 
     283=item force_upgrade 
     284 
     285Even if entry body already contains HTML, this config forces the 
     286plugin to upgrade the body. Defaults to 0. 
     287 
    283288=back 
    284289 

  • branches/feature-server/plagger/lib/Plagger/Plugin/Filter/FindEnclosures.pm

    r856 r937  
    8080 
    8181    # check $entry->link first, if it links directly to media files 
    82     $self->add_enclosure($args->{entry}, [ 'a', { href => $args->{entry}->link } ], 'href' ); 
     82    $self->add_enclosure($args->{entry}, [ 'a', { href => $args->{entry}->permalink } ], 'href' ); 
    8383 
    8484    my $parser = HTML::TokeParser->new(\$args->{entry}->body); 
    …… 
    110110    if (my $flashvars = first { lc($_->[1]->{name}) eq 'flashvars' } @params) { 
    111111        my %values = split /[=&]/, $flashvars->[1]->{value} || ''; 
    112         $url = first { m!^https?://! } values %values; 
     112        $url   = first { m!^https?://.*\flv! } values %values; 
     113        $url ||= first { m!^https?://.*! } values %values; 
    113114    } 
    114115 
    …… 
    149150            $content ||= $self->fetch_content($url) or return; 
    150151 
    151             if (my $enclosure = $plugin->find($content)) { 
     152            if (my $enclosure = $plugin->find({ content => $content, url => $url })) { 
    152153                Plagger->context->log(info => "Found enclosure " . $enclosure->url ." with " . $plugin->site_name); 
    153154                $entry->add_enclosure($enclosure); 
    …… 
    163164    my $ua  = Plagger::UserAgent->new; 
    164165    my $res = $ua->fetch($url, $self, { NoNetwork => 3 * 60 * 60 }); 
    165     return if $res->status != URI::Fetch::URI_OK && $res->is_error; 
     166    return if !$res->status && $res->is_error; 
    166167 
    167168    return decode_content($res);