[Ann] Google translate ruby client api 0.1 release.

I really like the google translate service. It’s very useful, especially to someone whose is not a native english speaker, such as me. However, I really hate to open a browser/tab to get a translation of some words. It’s no doubt that this work is time-consuming.
 Programmer is lazy, :)

So i wrote a ruby client api fro google translate service, then i can get the answer directly from the terminal. The world is much more beautiful. lol~

Install

sudo gem install rtranslate

Usage

sishen@lifegoo:~$ irb -rubygems
irb(main):001:0> require ‘rtranslate’
=> true
irb(main):002:0> $KCODE = ‘u’
=> "u"
irb(main):003:0> result = Translate.t("china", Language::ENGLISH, Language::CHINESE_SIMPLIFIED)
=> "中国"
irb(main):004:0> result = Translate.t("china", Language::ENGLISH, Language::JAPANESE)
=> "中華人民共和国"
irb(main):005:0> result = Translate.t("china", Language::ENGLISH, "zh-TW")
=> "中國"

Really very easy to use, I think. And the source is hosting on http://rtranslate.googlecode.com. Enjoy it.

[Postgresql] levenshtein distance support multibyte string

I don’t want to explain what the levenshtein distance mean, if you need some basic knowledge about this, check here levenshtein distance.

Postgresql have the build-in support for levenshtein distance which is written by Joe Conway,

Let’s see the results at first.

postgres=> select length(’阿’);
 length
——–
      1
(1 row)

postgres=> select levenshtein(’阿’, ‘俄’);
 levenshtein
————-
           3
(1 row)

Do you find anything strangely? The length of ‘阿’ is 1, it’s ok. It means postgresql have great support with the unicode/utf8 encoding. But the levenshtein result isn’t very friendly. From the above output, it’s clear that it can only support the ascii character, that’s why the result is 3. I think it disobeyed the design principle of postgresql.

I hacked the code and add the support of multibyte string. It works very well in my environment.

postgres=> select levenshtein(’阿’);
 levenshtein
————-
           1
(1 row)

postgres=> select levenshtein(’阿’, ‘俄’);
 levenshtein
————-
           1
(1 row)

You can download the patch here. Or if you are a debian user, i rebuild the contrib package with the 8.1.9 version. You can get it here.

If you have any problem, please contact me(yedingding@gmail.com), thanks.

[rubygem]A library to convert chinese to pinyin

前些日子, 用ruby写了一个汉字转化为拼音的类. 大家有用的话就自便吧, :).

原理很简单. 现在网上到处可以见到各种语种的实现, 原理都是大同小异. 简而言之, 先把汉字转化成GB2312编码, 然后查阅GB2312 Character Coding Table, 获取其对应的拼音码.

主代码:

      def to_pinyin(chinese)
        array = chinese.unpack("U*")

        pinyin = array.inject("") do |pinyin, integer|
          if isChineseUnicode(integer)
            pinyin += to_Pronunciation(integer)
          else
            # todo: other unicode
            pinyin += integer.chr
          end
        end
        pinyin.strip!
        pinyin
      end

 
由上面可以看出, 整个处理process是 chinese -> Unicode number -> iconv (ucs-2 -> gb2312) -> pinyin table, 具体的还是自己看代码吧, 很简单, :)

Test Case:

$:.unshift "../lib"

require ‘test/unit’
require ‘pinyin’

class ChineseToPinyinTest < Test::Unit::TestCase
  def test_pinyin
    assert_equal("zhong", Chinese::Pronunciation.to_pinyin("中"))
    assert_equal("ss", Chinese::Pronunciation.to_pinyin("ss"))
    assert_equal("zhongguo", Chinese::Pronunciation.to_pinyin("中国"))
    assert_equal(’womenenglishzifu’, Chinese::Pronunciation.to_pinyin(’我们english字符’))
    assert_equal(’zhonghuarenmingongheguo’, Chinese::Pronunciation.to_pinyin(’中华人民共和国’))
  end
end

Let’s run it:

$ rake
/usr/bin/ruby1.8 -Ilib:lib "/var/lib/gems/1.8/gems/rake-0.7.3/lib/rake/rake_test_loader.rb" "tests/ts_pinyin.rb"
Loaded suite /var/lib/gems/1.8/gems/rake-0.7.3/lib/rake/rake_test_loader
Started
.
Finished in 0.001338 seconds.

1 tests, 7 assertions, 0 failures, 0 errors

README:

= Pinyin
Turn chinese to pronunciation

This class enable ruby programs to turn chinese hanzi to its
pronunciation, that is, pinyin. It supports the GB2312 character
set coding table.

== Usage

require "pinyin"

pinyin = Chinese::Pronunciation.to_pinyin("xxx")

== CHANGES
v0.1.0 July 18
* initial release

Copyright (c) 2007 Dingding Ye <yedingding@gmail.com>
Distributed under MIT License

You can download the gem here.  I hope it’s helpful.

ansit.el : 语法着色FreeCity版

Freecity即浙江大学飘渺水云间BBS. Web做得越来越cool了, :)  当然, 本文所说的环境是telnet bbs. 文本的界面, 想搞点五颜六色的, 还要用ascii控制符, 对于我这种偶尔贴贴代码的, 着实让人累死. 可是, 既然是ascii控制符, 那么, 为啥不根据一些语法高亮的编辑器(如emacs)的效果而自动生成呢. 基于这个想法, 因为elisp, ansit.el诞生了.

首先, 感谢emacs, 他让整个世界更加美好, nothing is impossible, :)
再次, 感谢crazycool@smth, ansit.el是其于2004年为水木而写. 我在其基础上, 支持了freecity站点, 实际上, 只动了一点点代码… -,- 

效果:

下载:

ansit.el

PS:
配置方法: 以我的配置为例

(add-to-list ‘load-path (expand-file-name "~/EmacsLisp/")); EmacsLisp是我个人的elisp存放目录
(require ‘ansit)

使用方法: 具体参考源代码. 简单而言, 就是select the block and M-x ansit.   另外, ansit-ansify-this-in-new-buffer包含preview功能, 需要ansi-color lib, 一般当前的emacs版本都已自带(个人使用的是emacs-snapshot-unicode).

Google Reader Clip Client Code Review

我承认自己是Google的拥趸, :) 干净简洁的风格, 比较符合我的思想. Google Reader就是其一. 其上的附属服务, 也打上了深深的google烙印. Clip就是其一.

Why Google Reader Clip? Google Reader lets you tag blogs and blog posts and share those items with your friends. 简而而已, 可以把你的Feed items集成到你的个人网站里. 贴心的小服务, 又是很好的宣传手段, cool~

Howto? Customize Google Reader's Clip. 还有很多教程, google吧...

最后得到的html snippet就是:

本文的目的在于分析这段代码是如何工作的.

首先, 关注两个URL.

• http://www.google.com/reader/ui/publisher.js
• http://www.google.com/reader/public/javascript/user/01965866220529097173/label/developerworks?n=5&callback=GRC_p({c:\"green\",t:\"\\\"developerworks\\\" via sishen\",s:\"false\"});new GRC

publisher.js是被混淆过的JS, 其职责在于提供提供默认clip显示函数库. 看得我累死了... >_<
下一个URL则是整个clip的生成脚本. 如果你细心的话, 就会发现query string里面有个callback参数. 很精妙的设计, :) 通过callback参数值, 我们可以自主控制clip的显示方式(customize, 见前面的link). 当然, 现在默认使用的是
GRC_p({c:\"green\",t:\"\\\"developerworks\\\" via sishen\", s:\"false\"}); new GRC. 而这个就是publisher.js里的object.

那么, 下面的主题就是publisher.js了. 一段很短小精悍的程序(其中50%是浏览器判断+异常处理). 整个核心在于GRC object. 在介绍GRC object前, 有一点需要说明的是在clip里的item是通过http: //www.google.com/reader/public/javascript/user/:id/label/lablename获得的. 至于怎么获得的, 不是我所关心的. 最后的形式就是
GRC_p({c:"gray",t:"\"developerworks\" via sishen",s:"false"});new GRC({--item description--}).
GRC_p里面的是参数配置, c为背景色, t为标题, s为显示item来源.

对应的代码为:

function ca(a)
{
  u=a
}
function readerpublisher(a,b)
{
  this.id="readerpublishermodule"+this.version();
  this.g = a;
  this.o = u["c"];
  this.p = u["t"];
  this.J = u["s"]=="true";
  if(b)
  {
    b.innerHTML="";
    b.id=this.id;
    this.createTN(getElemById(this.id))
  }
  else
  {
    document.write('');
    var c=this;
    window.setTimeout(function() {c.createTN(getElemById(c.l))}, 0)

  }
}

GRC({})则根据items来生成相应的html代码. item格式为:

"id": "user/01965866220529097173/label/developerworks",
"title": "u0022developerworksu0022 via sishen in Google Reader",
"continuation": "CMfBw7COhIwC",
"author": "sishen",
"updated": 1178866000,
"items": [
{
"id": "tag:google.com,2005:reader/item/bc6a9f32df0c289b",
"categories": [
],
"title": "u00E5u0153u00A8 Linux on POWER u00E4u00B8u0160u00E8u00BFu203Au00E8u00A1u0152u00E5u00BAu201Du00E7u201Du00A8u00E7u00A8u2039u00E5u00BA u00E5u00BCu20ACu00E5 u2018",
"published": 1178866000,
"updated": 1178866000,
"alternate": {
"href": "http://www.ibm.com/developerworks/cn/linux/linux_app.html?cau003Ddrs-tp2007",
"type": "text/html"
},
"summarySnippet": "u00E5u0153u00A8 IBM System pu00E2u201Eu00A2 u00E5u2019u0152 System iu00E2u201Eu00A2 POWER u00E5u00B9u00B3u00E5 u00B0u00E4u00B8u0160u00E5u00BCu20ACu00E5 u2018u00E5u2019u0152u00E9u0192u00A8u00E7u00BDu00B2 Linuxu00C2u00AE nu00E5u00BAu201Du00E7u201Du00A8u00E7u00A8u2039u00E5u00BA u00E4u00B8u017Du00E5u0153u00A8u00E5u2026u00B6u00E4u00BBu2013 Linux u00E7u00B3u00BBu00E7u00BBu0178u00E4u00B8u0160u00E7u0161u201Eu00E5u00BCu20ACu00E5 nu2018u00E5u2019u0152u00E9u0192u00A8u00E7u00BDu00B2u00E7u203Au00B8u00E4u00BCu00BCu00E3u20ACu201Au00E5u0153u00A8u00E6u0153u00ACu00E6u2013u2021u00E4u00B8u00ADu00EFu00BCu0152u00E8u00AEu00A8u00E8u00AEu00BAu00E5u00AFu00B9u00E4u00BAu017D Linux on POWER nu00E7u00B3u00BBu00E7u00BBu0178u00E5u00BAu201Du00E8u00AFu00A5u00E4u00BAu2020u00E8u00A7u00A3u00E7u0161u201Eu00E7u203Au00B8u00E4u00BCu00BCu00E6u20ACu00A7u00E5u2019u0152u00E5u00B7u00AEu00E5u00BCu201Au00E3u20ACu201A",
"origin": {
"streamId": "feed/http://www.ibm.com/developerworks/cn/rss/dwcn_dwtp.rss",
"title": "IBM developerWorks u00E4u00B8u00ADu00E5u203Au00BD",
"htmlUrl": "http://www.ibm.com/developerworks/cn/index.html?cau003Ddrs-tp2007"
}]
}

ok. 切入主题. 整个clip的生成过程就是createElement和createTextNode的过程.

function createElement(a,b)
{
  var c=document.createElement(a);
  if(b)
  {
    for(var d in b)
    {
      var f=b[d];
      if($.q()&&d=="class")
      {
         d="className"
      }
      c.setAttribute(d,f)
   }
 }
 return c
}

function createTextNode(a)
{
 return document.createTextNode(a)
}

看一下例子:

var h3_elem =createElem("h3");
this.merge_style(c.H, h3_elem);
d.appendChild(createTN(this.p));
a.appendChild(h3_elem);

其显示效果就是:

<h3 style="i hidden the style... it's too long">"developerworks" via sishen</h3>

接下去是一个大循环, 遍历所有item.

var f=createElem("ul");
for(var F=0,m;m=this.document.items[F];F++)
{
}

具体的为:

var w=createElem("li");
var G=createElem("a",{href:link(m[alternate].href),title:m[title],"class":"i"});
var h=strip_tag(m[title]);
if(!(strip(h)==""))
{
   h=fa(h,48)
}
G.appendChild(createTN(h));
w.appendChild(G);
f.appendChild(w)

最后的生成效果为:

<a style="i hidden the style... it's too long" class="i" title="Make a List and Check It Twice: A Real-World Guide for Speakers and Presenters" href="http://feeds.feedburner.com/%7Er/guykawasaki/Gypm/%7E3/115725056/make_a_list_and.html">Make a List and Check It Twice: A Real-World Guide...

基本整个publisher.js的核心就在于上面了. 当然, 还有很多辅助函数, 有兴趣的就自己看吧, :)

Use ant to send file to remote automatically

最近一直在做测试, 需要把自己的程序扔到远程服务器上执行. 于是, 在起初的阶段, 我每次都非常痛苦地打一堆字”scp -P xx file user@host:/some/dir”, 让人接近崩溃.. -,-

痛定思痛, 决定来自动化这个过程. 所以直接把目光投向了Ant, 一个强大的构建工具. Ant是伟大的, :) 它提供了remote task, 支持通过ftp/scp/sshexec等任务来完成远程操作. 我下面将针对scp task.

Scp task是Ant的一个optional task, 所以要运行这个task, 首先需要解决dependencies — jsch(java对ssh2的实现). 注: jsch需要0.1.29以上版本. 可以去Jsch主页下载.

关于Scp task的每个parameters的用法, 我这里就不累赘了. 可以去Ant主页上查看. 我下面就给出几个简单的例子.

Copy a remote directory to a local directory

<scp file=”user:pass@host:/home/sishen/*” todir=”/home/sishen”/>

Copy a single local file to a remote machine with separate password attribute

<scp file=”file” todir=”user@host:/home/sishen” password=”pass”/>

Copy a single local file to a remote machine using key base authentication.

<scp file=”myfile.txt”
     todir=”user@somehost:/home/chuck”
     keyfile=”${user.home}/.ssh/id_dsa”
     passphrase=”my extremely secret passphrase”/>

在这基础上, 你还可以用FileSet, DirSet, FileList来group要传送的文件. 比如

<scp todir=”xxx@xxx:~/xxx”
     port=”22″
     keyfile=”${user.home}/.ssh/id_rsa”
     passphrase=”xxxx”>
     <fileset dir=”lib”>
        <include     <exclude </fileset>
     <fileset dir=”${build.dir}”>
        <include </fileset>
</scp>

这里也引进了一个问题. 密码是以明文的形式直接显示的, 而这却是我们无法接受的. 针对这个, 我们有3种解决方法.

1. 通过command argument来输入密码. 比如ant -Dpassphrase=xxxxx.
2. 把passphrase=xxx存储在文件里, 设置权限为600. 然后ant -propertyfile 来获取密码.
3. 通过input task. 由用户交互输入密码.

从 安全性考虑, 只有方法3才是让我满意. 然而, 使用方法3, 你又将面临一个问题, 密码输入过程是明文的. 这是则是InputHandler的决策. 当不指定InputHandler, ant将使用DefaultInputHandler, 而DefaultInputHandler则是读取用户输入, 然后显示在console上. 那么, 解决的方法也就明了了. 我们需要实现一个MaskPasword的InputHandler. 关于这个, 已经有实现了. 其中心思想是设置InputStream为System.err(即取消buffer), 然后读取用户输入,读到一个就用\b*(backspace+*)来代替.

下面是我的一个完整的应用. 我用了单独的target.

<target depends=”jars” description=”send file to remote server”>
<echo level=”warning” message=”send file through ssh ing”/>
<input message=”Enter passphrase for key ‘${user.home}/.ssh/id_rsa’” addproperty=”passphrase”/>
<scp file=”${build.dir}/${app.jar}”
     todir=”user@host:~/dir/”
     port=”22″
     keyfile=”${user.home}/.ssh/id_rsa”
     passphrase=”${passphrase}”
     trust=”yes”/>
</target>

这里有一点要注意, 就是trust=”yes”. 这个是表示接受所有unknown的host. Jsch只支持SSHV2的known_host. 而Openssh在knows_host文件里, 对host进行了hash, 所以造成每次匹配不成功, 因而使用openssh的客户端, 必须得加上trust=”yes”.

自此以后, 轻松了. 只要运行ant remote -inputhandler org.apache.tools.ant.input.PasswordInputHandler, 输入密码, 所以要传送的文件就会乖乖的到远方去了… 如果嫌麻烦的话, 把remote设为default…

Resource:

• extendinput.jar: PasswordInputHandler的jar包

在线协同开发, 你用什么?

在软件开发工程中, 如何提高协同开发的效率是关键. 一个好的平台, 给生产力带来的提高绝对不止一倍两倍. 网络的普及, 使得传统的会议式协同合作, 已经满足不了需求, 在线协同开发逐渐成为了项目管理的主流方式. 尤其是Open Source的流行, 给协同开发带来了很多新的模式, 也出现了很多优秀的开源在线协同开发工具.

在我看来, 一个合适的协同开发平台, 至少应该包括source control system, issue tracking system, bug tracking system, 还有是团队的交流平台.

SVN/CVS作为SCM, 为协同开发提供了很好的代码控制系统. 基本上, 在VC方面, 这个不大有改变.

很多时候, issue tracking system和bug tracking system可以重用(当然, 从细致上讲, 独立为好). 一个bug也算是一个defect issue. 目前, 应用比较广泛的如Bugzilla, trac, roundup都是不算的工具.

交流平台也有很多工具. 从原始的EMail, 到Maillist/Group/Forum/IM, 进而Wiki/IRC, 都是不错的创新. Maillist/Wiki之类, 缺少实时性; IM/IRC则缺少点沉淀. 如何合理使用也是一门学问.

推荐一个工具, 呵呵. 最近一直在用trac. 觉得是一个很不错的在线协同开发工具. 它集成了wiki/issue tracking system/svn. 使用简单, 功能强大. 有兴趣的自己google,

总之, 一切用了程序员的偷懒,

在线协同开发, 你用什么?

ps: 有些online的在线协同开发平台, 如SourceForage, Basecamp, 也是不错的选择喔,