Excel date_format tostring string.format type function

Posted: March 8th, 2011
By:
admin

So it took me about 10 minutes to figure out that Excel has a date_format, date, date.toString() like function. It’s called text(). To get a date format into something which will import nicely into a postgres or mysql database, simply do somthing like this:
text(A1, "yyyy-mm-dd")

And since all the other languages seem to have a more clear “date format” function, here are the corresponding methods:

  • .Net/C# – String.Format() or DateTime.ToString()
  • php – date()
  • mysql – date_format()
  • postgres – to_char()
  • java – String.Format combined with a java.util.Calendar

Copy data from local file with Postgres and psql

Posted: February 8th, 2011
By:
Tommy Unger

It was surprisingly hard to find a functional, working example of copying data from a file not based on the server into a PostgreSQL database.

cat file.txt | psql -c “COPY table_name FROM STDIN” -U username database_name

And don’t forget the handy trick for disabling the password prompt in psotgres.

A few helpful search terms:

  • Copy file into Postgres from local filesystem
  • Copy local file into Postgres from command line
  • Using psql to copy file into Postgres database from file system

Running postgresql from command line without password prompt

Posted: November 30th, 2010
By:
Tommy Unger

Due to security reasons, postgres (in contrast to mysql) does not allow a user to specify the password on the command line. However, there is a quick and hacky workaround which allow a psql command to be run without the password prompt.

From the command line, simply type:
export PGPASSWORD=yourpassword

From there you can now run:
psql -U username database

Pulling All Database Table Sizes from PostgresSQL

Posted: June 10th, 2010
By:
Tommy Unger

I was surprised that I didn’t easily find a query which gives me all of the table sizes for all of my Postgres database table. I did find this nice bit:
SELECT pg_size_pretty(pg_total_relation_size(table_name))

But I wanted to take it a step further and get the list of the largest tables in my database. This query worked like a charm:
SELECT table_name, pg_size_pretty(pg_total_relation_size(table_name))
FROM information_schema.tables
WHERE table_type='BASE TABLE'
AND table_schema = 'public'
ORDER BY pg_total_relation_size(table_name) DESC

Finally, I wanted to know if any of these tables had a column name containing the text “org id”.
SELECT t.table_name, pg_size_pretty(pg_total_relation_size(t.table_name))
, case when cn.table_name is not null then ‘Yes’ ELSE ” end as “has_col”
FROM information_schema.tables t
LEFT JOIN
(select distinct c.table_name
from information_schema.columns c
where c.column_name LIKE ‘%org%id%’) cn on cn.table_name = t.table_name
WHERE t.table_type=’BASE TABLE’ AND table_schema = ‘public’
ORDER BY pg_total_relation_size(t.table_name) DESC
;

Configuring MySQL service on Windows to use a different my.ini file

Posted: December 1st, 2009
By:
Tommy Unger

MySQL 5 conveniently comes with a windows installer which works flawlessly to get the novice user up and running on the ubiquitous open source database. However, another way to put mysql onto a windows box is without the installer by just downloading the zipped binaries.

Once you’ve unzipped the binaries, you have a few steps to take:
Step 1. Run mysqld –install – This will setup the mysql daemon as a windows service. It will look in your windows base directory %WINDIR% for a my.ini file. (On windows the my.conf file is called my.ini, and I have no idea why this is)

Step 2. Open regedit. And change this key
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\MySQL\ImagePath to this:
“<path-to-mysql>\bin\mysqld” –defaults-file=”<path-to-mysql>\my.ini” MySQL
or this, if you just want to use one of the default medium sized instance ini file:
“<path-to-mysql>\bin\mysqld” –defaults-file=”<path-to-mysql>\my-medium.ini” MySQL

Email Spam and an argument for gmail

Posted: July 16th, 2009
By:
Tommy Unger

This post Survey: People really do click on spam ads really got me thinking about a few things. First, my history.

Back in some year, let’s say 1998 or maybe 1997 I got my first portal email address (had an aol address and a college address before that). It was tommy_unger@yahoo.com . Yes, I’m putting the email right out there for all of the bots to parse, and I expect the spam onslaught continue. The address is absolutely worthless because it is 99% spam! Much of it in foreign languages (arabic or persian or something from that part of the world seems popular these days. Then of course there’s the cryptic meaningless text, or just “Hi, come to my website so we can have sexy time”. My 8 month old son could probably detect spam better than Yahoo’s algorithms.

Then, finally, came gmail. I signed up with a better address. Same name as above without the underscore, and low-and-behold, I see about one spam message a month. I remember seeign a video a few years back about gmail’s spam detection and it seems right on the money. What I also like is that gmail also rarely(maybe never) has the false positive case where they dump an important message into the spam bucket.

So, back to the article I began with… It mentions:

few users ever click the “report as spam” button on their email client; most just delete spam messages as they arrive (and fail to help train their spam filters in the process)

I’ve got a hunch that gmail has higher rates of this “report as spam” click than the other email clients. I also have a feeling that yahoo mail’s “report as spam” button is just a black hole that ever gets used to clean up anyone’s spam.

Combine gmail’s possible higher “report as spam” click rate with a probable much better algorithm and/or system and you get a much nicer web mail experience in my opinion. So, thanks yahoo tech blog (you still do content well, of course), for reminding me why I love gmail.

Recursively remove subversion (.svn) directories

Posted: March 24th, 2009
By:
Tommy Unger

I don’t care if this is a nerdy post or if it’s duplicated all over the internet. I just want to put it out there for reference. Here’s the linux/unix/cygwin code for recursively removing subversion directories.

rm -rf `find . -name .svn`

I found the example here, but you can never have too much duplicate content floating around the internet.

#000000
#1d3147
#415973
#9dacb9
#4c3724
#897661
#d1c8c0
#ffffff