La Vita è Bella

I wrote a vim plugin named projtags, initially for loading tags file for projects. But as I can "set tags+=tags;" in my vimrc, it's not that useful on the tags file way. I expanded its feature to support commands for per projects, and this is much more useful Here are some examples:

Case 1: different coding styles

I did some squid hacking before. My coding style is as below:

set cindent
set autoindent
set smartindent
set tabstop=8
set softtabstop=8
set shiftwidth=8
set noexpandtab
set smarttab
set cino=:0,g0,t0

But squid coding style is different. They use 4 for shiftwidth. As hacking some project, you should always follow the original coding style. But add vim modeline into every file is also unacceptable. So I use projtags.vim to do this job:

let g:ProjTags += [["~/work/squid", ":set sw=4"]]

So that every time I'm editing a file within squid, vim will use 4 instead of 8 for shiftwidth, now I'm happy with both my own codes and squid codes.

Case 2: use ant as makeprg for java projects

First, I use a autocmd to set ant as makeprg for java files:

autocmd BufNewFile,BufRead *.java :setlocal makeprg=ant\ -s\ build.xml

It works well for java files. But as I did some Android development these days, I also have lots of xml files to edit (and then make to see the result). I can't use such a autocmd for xml's, as not all xml's are in a java project. So projtags.vim is again the answer:

let g:ProjTags += [["~/work/pmats", ":set mp=ant\\ -s\\ build.xml"]]

So I can always use ":make" for ant under my Android project now, no matter it's .java or .xml.

My vimrc segment for tags and projtags

set tags+=/usr/local/include/tags
set tags+=/usr/include/tags
set tags+=/opt/local/include/tags
set tags+=tags;

let g:ProjTags = []
let g:ProjTags += [["~/work/squid", ":set sw=4"]]
let g:ProjTags += [["~/work/pmats", ":set mp=ant\\ -s\\ build.xml"]]

Patches for projtags.vim (e.g. Windows support, I haven't tested it but I guess it won't work under Windows now) are welcomed You can get the code from the git repo for my various scripts.

Columbus V-900 is so far the GPS that best fit my requirements: it can log, it can take waypoints while logging, and it (can) have a big storage for logging (via TF card). I got one from WC, a friend to give it a try.

It log GPS tracks to CSV format, WC wrote a script to convert it to the GPX format, but without waypoints. So I rewrote a Python script (as I'm not so familiar with perl), to add the waypoints to the GPX file.

Get the script here, it will use the voice record filename as the name of the waypoint if available, or otherwise just "Waypoint #N". You may want to edit the converted GPX file to rename the waypoints.

This script will convert a line contains start and end IP separated by space like:

221.192.0.0 221.199.207.255

Into IP mask format (with comment) like:

#221.192.0.0 - 221.199.207.255
221.192.0.0/14
221.196.0.0/15
221.198.0.0/16
221.199.0.0/17
221.199.128.0/18
221.199.192.0/20

The script is:

 1 #!/usr/bin/env python
 2
 3 import sys
 4 import re
 5
 6 def ip2int(ip) :
 7         ret = 0
 8         match = re.match("(\d*)\.(\d*)\.(\d*)\.(\d*)", ip)
 9         if not match : return 0
10         for i in xrange(4) : ret = (ret << 8) + int(match.groups()[i])
11         return ret
12
13 def int2ip(ipnum) :
14         ip1 = ipnum >> 24
15         ip2 = ipnum >> 16 & 0xFF
16         ip3 = ipnum >> 8 & 0xFF
17         ip4 = ipnum & 0xFF
18         return "%d.%d.%d.%d" % (ip1, ip2, ip3, ip4)
19
20 def printrange(startip, endip) :
21         bits = 1
22         mask = 1
23         while bits < 32 :
24                 newip = startip | mask
25                 if (newip>endip) or (((startip>>bits) << bits) != startip) :
26                         bits = bits - 1
27                         mask = mask >> 1
28                         break
29                 bits = bits + 1
30                 mask = (mask<<1) + 1
31         newip = startip | mask
32         bits = 32 - bits
33         print "%s/%d" % (int2ip(startip), bits)
34         if newip < endip :
35                 printrange(newip + 1, endip)
36
37 while 1 :
38         line = sys.stdin.readline().strip()
39         if not line : break
40         chars = line.split(" ")
41         print "#%s - %s" % (chars[0], chars[1])
42         ip1 = ip2int(chars[0])
43         ip2 = ip2int(chars[1])
44         printrange(ip1, ip2)
45

The emails sent by NucleusCMS (e.g. new comment notification) didn't encode the subject, but just put the raw text there. For english language file, that's OK. But for other languages such as Chinese, it's not that good. It will produce garbled text on the subject (but not always garbled, weird).

So I wrote this patch to resolve this problem. Google tell me that if I want to encode an email subject in PHP, I should use the mb_encode_mimeheader() function. But Dreamhost just didn't provide any mb_ functions in PHP. I dug more about email subject encoding, and found that a hardcoded base64 should just do the trick. As the original sending mail code in NucleusCMS hardcoded UTF-8 in the Content-Type, I assume that hardcode UTF-8 in the subject is fine, too.

My company uses svn, and a VPN without Mac client. So I can't commit from home. In order to manage my off-time works, I use git-svn on my MacBook Pro.

Today I'm going to sync my 2 commits in git to svn, using:

$ git svn dcommit

But after committed the first change, the svn server have no spaces left! So the second commit was failed:

End of file found: Can't read file '/tmp/report.tmp': End of file found at /usr/bin/git-svn line 3856

After svn admin resolved this problem, and I dcommit again, but it still fails:

Merge conflict during commit: File or directory '***.cpp' is out of date; try updating: The version resource does not correspond to the resource within the transaction. Either the requested version resource is out of date (needs to be updated), or the requested version resource is newer than the transaction root (restart the commit). at /usr/bin/git-svn line 461

And I've tried

$ git svn fetch

but the problem remains. So I decided to manually commit that change.

First, generate a patch from git ("HEAD^" means make a patch against the parent of HEAD, so it's my last change):

$ git format-patch HEAD^

And this command will generate a 0001--***.patch file. Then, use "svn co" to checkout a svn working copy, and apply that patch:

$ patch -i /path/to/0001--***.patch -p1

Then, get my last commit log from "git log", then use "svn ci" to commit it.

Now back to the git working copy, rebase svn:

$ git svn fetch
$ git svn rebase

Then try dcommit again, no conflicts and no commit made (as the svn repository is already up to date).

Hooray!

There's a macro in ACE to eliminate a compiler warning:

379 // Some compilers complain about "statement with no effect" with (a).
380 // This eliminates the warnings, and no code is generated for the null
381 // conditional statement.  @note that may only be true if -O is enabled,
382 // such as with GreenHills (ghs) 1.8.8.
383 # define ACE_UNUSED_ARG(a) do {/* null */} while (&a == 0)

But when I use this macro, gcc 4.3 will complain:

warning: the address of ‘a’ will never be NULL

There're 2 versions of iconv.h in which the iconv() have different prototypes.

In Debian, it's:

43 extern size_t iconv (iconv_t __cd, char **__restrict __inbuf,
44                      size_t *__restrict __inbytesleft,
45                      char **__restrict __outbuf,
46                      size_t *__restrict __outbytesleft);

And in other systems, such as Mac OS X, it's:

83 extern size_t iconv (iconv_t cd, const char* * inbuf, size_t *inbytesleft, char* * outbuf, size_t *outbytesleft);

So you can see that the second parameter, "inbuf", can be "const char **" or "char **"

It will make a big problem while writing code for both systems. One resolution is to use condition compile ("#ifdef" blah blah...), but RoachCock@newsmth give me another (and much better) resolution: use operator overload (so it's C++ only):

133 struct iconv_param_adapter {
134         iconv_param_adapter(const char**p) : p(p) {}
135         iconv_param_adapter(char**p) : p((const char**)p) {}
136         operator char**() const
137         {
138                 return (char**)p;
139         }
140         operator const char**() const
141         {
142                 return (const char**)p;
143         }
144         const char** p;
145 };

When you calling "iconv()", call it like this:

111         size_t res = iconv(data, iconv_param_adapter(&s), &inbytesleft, &outnew, &outbytesleft);

Thank you RoachCock!

expat is a good XML parser, light and quick. But it only support latin1, UTF-8 and UTF-16 naturally, if you want to use it to deal with other encoding XML's, you need to set a unknown encoding handler.

I use libiconv to convert the GBK string to Unicode for expat.

First, implement a function to pass to XML_SetUnknownEncodingHandler:

int
XML_Stream_Parser::xml_unknown_encoding(void* data, const char* name, XML_Encoding* info) {
        iconv_t cd;
        if(strncasecmp(name, "GB", 2) != 0 || (cd = iconv_open("UCS-2BE", name)) == (iconv_t)-1) {     // not GB, unsupported
                fprintf(stderr, "can't convert %s\n", name);
                return 0;
        }
        for(size_t i=0; i<128; i++) info->map[i] = i;
        for(size_t i=128; i<256; i++) info->map[i] = -2;
        info->convert = XML_Stream_Parser::xml_convert_gb;
        info->release = XML_Stream_Parser::xml_convert_release;
        info->data = cd;
        return 1;
}

In this function, I tell expat that for GBK encoding, ASCII 0~127 is left as is, and ASCII 128~255 will need to be dealt together with the next byte.

Then implement the "convert" and "release" functions:

int
XML_Stream_Parser::xml_convert_gb(void* data, const char* s) {
        const size_t out_initial = 4;
        size_t inbytesleft = 2, outbytesleft = out_initial;
        char *out = new char[out_initial], *outnew = out;
        size_t res = iconv(data, &s, &inbytesleft, &outnew, &outbytesleft);
        int ret = 0;
        if(res == (size_t)-1) {
                fprintf(stderr, "error in conversion\n");
                delete []out;
                return '?';
        }
        for(size_t i = 0; i < out_initial - outbytesleft; i++)
                ret = (ret<<8) + (unsigned char)out[i];
        delete []out;
        return ret;
}

void
XML_Stream_Parser::xml_convert_release(void* data) {
        iconv_close(data);
}

In "convert", I use iconv to convert the string to unicode, and return the unicode to expat.

The limitation in this interface is that it can't deal with 4-byte GB18030 codes, as I can't judge whether it's a 4-byte code just by the first code.

Anyway, I suggest that all XML should be encoded to UTF-8, so that this is unneeded

I need some efficient way for Miao for word database, and I guess SQLite is the solution. So I wrote this simple test program to get familiar with SQLite API. (read more for the program)

And there's something should be kept in mind: don't use "`"s.

As I used to use MySQL before, I always put "`" in SQL statements, for example:

create table `foo` (`bar` int)

It's OK in SQLite command line program, but not in the C library.

When I execute a SQL query with "`", I'll get this error message:

sql error #1: unrecognized token: "`"

So I deleted "`"s in the SQL statement, but still get this error message:

sql error #4: malformed database schema - unrecognized token: "`"

This really drive me crazy, as I don't know what's wrong. But finally I've got the reason: that's because there's "`" in the "create table" statement. So I recreated the table and rerun the program, it's finally OK now.

more...

If you use vim for programming, then you can't live without ctags (can you?). ctags generate the "tags" file, vim and its plugins use this file to help your programming more efficiently.

By default, vim will only use the "tags" file under your current working directory. You can use this command to see it:

:set tags?

Generally, your "tags" file under you current working directory won't contain informations about system libraries (glibc, stl, etc.). If you miss them, you can generate tags file for your system libraries, and ask your vim to load them:

set tags+=/usr/local/include/tags
set tags+=/usr/include/tags

If you are working on a big project, which have many subdirectories, the "tags" file under each working directory may be not enough, as you also need some information about the functions under other subdirectories. So you can generate a "tags" file under your project root, and ask vim to load it when editing a file within your project:

autocmd BufEnter ~/work/myproj/* :setlocal tags+=~/work/myproj/tags

Thanks for Ryan Phillips, you can also add "tags;" (notice the semicolon) to your tags so that vim will automatically look up "tags" file in the file tree (":help file-searching" for document):

set tags+=tags;

To make it easier to use, I've also made a script named "projtags.vim" so that you need only set your project directories in your vimrc:

let g:ProjTags = ["~/work/proj1"]
let g:ProjTags+= [["~/work/proj2", "~/work/proj2.tags"]]

Happy vimming