Sunday, April 28, 2019

tmux

list sessions
tmux ls
attach
tmux a name
detach
ctrl+b d
ctrl+b a name
split horiz
ctrl+b "
split vert
ctrl+b %
console
ctrl+b :
cycle panes
ctrl+b o
move to another:
ctrl+b [arrow-key]

Monday, July 17, 2017

HIVE load csv text file

First column must not contain spaces, otherwise it will load NULL! Intermediate columns can contain spaces, it is suggested to load them as String and then convert to the required type.
1.Load into special text table
CREATE TABLE tt (
  id BIGINT,
  data STRING
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE
2.Move to the normal table

Tuesday, May 9, 2017

Hive QL vs SQL


No "IN" Clause
Zero-based indexing
"AS" aliasing for columns only


SQL





HIVE QL
ORDER BY
SELECT * FROM t WHERE id in (...)
SORT BY (ORDER BY is partial sort)
SELECT * FROM t LEFT SEMI JOIN t2 ON ...
SELECT f[1] FROM t SELECT f[0] FROM t
SELECT * FROM t AS tt                     SELECT * FROM t tt


Tuesday, May 2, 2017

Mercurial commands

Create a branch
hg bookmark [name]
Reset local changes
hg revert --all
Fetch latest code
hg pull
Fetch latest code and update local code
hg update master
Commit
hg commit [-m "message"]
Update commit with new changes
hg amend [!! use of -m "message" overwrites the log]
Status
hg diff
hg status
Current bookmark (branch)
hg identify
Switch bookmark and rebase
hg update [name]
Difference from the rev
hg diff -r [rev_num]
Use file from a revision
hg revert -r [rev_num] [file]
Use theirs or yours version on rebase
hg resolve -t internal:other --all
hg resolve -t internal:local --all
Use manual resolve on rebase

hg resolve -m [file]



Friday, September 16, 2016

Integer overflow

Integer overflow in JVM can lead to negative, zero and positive values, i.e. can everything:
scala> val x = 1 + (Int.MaxValue - 1) / 2
x: Int = 1073741824
scala> x * x
res16: Int = 0
scala> val x = 1 + (Int.MaxValue - 5) / 2
x: Int = 1073741822
scala> x * x
res18: Int = 4
scala> val x = 1 + (Int.MaxValue - 6) / 2
x: Int = 1073741821
scala> x * x
res19: Int = -2147483639
The check for overflow due to multiplication divide back and check if get the same result. Summation overflow can be checked by the sign of result. The other option is to use double and long. 

Thursday, August 4, 2016

Check scalastyle

Maven - checks only main sources:
mvn scalastyle:check
Sbt check main:
sbt scalastyle
Sbt check test:
sbt test:scalastyle