Hive UDF - Convert Date to Unix Timestamp Example

Hive UDF - Convert Date to Unix Timestamp

Background

This little UDF will convert a date, in any specified format, into a unix timestamp.  To specify the date just edit the string in the SimpleDateFormat to your liking. So here is how we did it.  

I have also left in the imports and you will need to find the jar files that contain these classes.  

Implementation

package allan.DtoT;
import java.text.ParseException;
import java.text.SimpleDateFormat;

import org.apache.hadoop.hive.ql.exec.UDF;

public class DateToTime extends UDF{
 public long evaluate(final String d){
  try{
   SimpleDateFormat sf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
   return sf.parse(d.trim()).getTime();
  } catch (ParseException pe) {return -1;}
  
 }
}

Pig UDF Example

Pig UDF Example

Background

This little UDF will convert the first 8 bytes of an HBase key into a long.  The Key that we had was a composite key made up of two 8 Byte longs and we needed to convert the first 8 bytes and then the second 8 bytes to get them separately. So here is how we did it.  

I have also left in the imports and you will need to find the jar files that contain these classes.  

Implementation

package allan.myudf;
import java.io.IOException;

import org.apache.hadoop.hbase.util.Bytes;
import org.apache.pig.EvalFunc;
import org.apache.pig.backend.hadoop.hbase.HBaseBinaryConverter;
import org.apache.pig.data.DataByteArray;
import org.apache.pig.data.Tuple;


public class ConvertFirst extends EvalFunc<Long> {
 public Long exec(Tuple input) throws IOException {
  if (input != null && input.size() == 1) {
   try {
    DataByteArray a = (DataByteArray) input.get(0);
    HBaseBinaryConverter b = new HBaseBinaryConverter();
    return Bytes.toLong(b.toBytes(a),0,8);
   

                        } catch (IllegalArgumentException e) {
    System.err.println("...");
   }
  }
  return null;
 }
}