cursethiscure
2012-Jul-19 17:23 UTC
[R] Switching log(J) to log(J+1) to avoid log(0) in HAR-RVJ model
I am working with xts dependent data, and my code is as follows (the problem
is explained throughout):
dat <- getdat("prices")
dat <- read.zoo(dat, sep = "",format="%d/%m/%Y %H:%M",
tz="", FUN=NULL, regular=TRUE,
header=TRUE, index.column=1, colClasses=c("character",
"numeric"))
dat <- as.xts(dat)
## cleaned data here to get
daylist <- lapply(split(dat, "days"), function(x) {
if(NROW(x) >= 10) x
})
do.call(rbind, daylist) -> dat
makeReturns(dat) -> dat
the code I am running to get the HAR regressions (full code for it shown
below) is
x = harModel(dat, periods = c(1,5,22), periodsJ=c(1), RVest
c("RCov","RBPCov"),
type="HARRVJ", h=5, transform="log") ; #
Estimate the HAR model
of type HARRVJ
The three HAR models on paper are:
1.?RV?_(t,t+h) = ?_0+ ?_D ?RV?_t+ ?_W ?RV?_(t-5)+ ?_M ?RV?_(t-22) + ?_J J_t
+ ?_(t,t+h) # NULL model in code at bottom
2. (RV?_(t,t+h) )^(1/2) = ?_0+ ?_D ??RV?_t?^(1/2)+ ?_W (?RV?_(t-5) )^(1/2)+
?_M (?RV?_(t-22) )^(1/2) + ?_J (? J?_t )^(1/2) + ?_(t,t+h)
3. log(RV?_(t,t+h) )= ?_0+ ?_D.log(?RV?_t )+ ?_W.log(?RV?_(t-5) )+
?_M.log(?RV?_(t-22) ) + ?_J log(J_t + 1) + ?_(t,t+h)
Basically the harModel in the code allows you to transform the regressions
from NULL to "sqrt" or "log", but when
`transform="log"` is chosen it gives
the following error message.
x = harModel(dat, periods = c(1,5,22), periodsJ=c(1), RVest
c("RCov","RBPCov"),
+ type="HARRVJ", h=22, transform="log") ; #
Estimate ....
[TRUNCATED]
Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) :
NA/NaN/Inf in foreign function call (arg 1)
which is due to it taking the log(0), the actual log model should take
log(J + 1) in case of a 0 value for the J in the time series ( if J does not
occur it should return a 0, otherwise it should return J), but unfortunately
I do not know how to rectify this. I was wondering if any one could tell me
how how I can achieve this as I am very naive with R still. I have tried
some modifications you will see at the bottom of the question, but they do
not compute the regression correctly though they do allow it to run without
'error'
A sample of the xts returns are
structure(c(0, 0.00163990674377068, -0.000417175726423302,
-0.000536624643948436,
-0.000417573901733803, -0.000656540057603294, -0.000119417243992004,
0.000149269326922941, 0.000865322983560901, 0.000387660473923468,
-0.000626295482812012, -0.000477440925755523, 0.00080554938345756,
-0.000835397011320183, -2.98485187695263e-05, 5.96961466321488e-05,
-0.000388088668204389, -0.00062723079668725, -0.00230321804403655,
-0.00206843906653553, -0.0013212421596549, 0.0001802722115265,
0.000510594854085333, -6.00564530843783e-05, -0.000150156914257238,
0, 0.000180185591646342, 0.0012004082829149, 0.00179791490358649,
-2.99387752074054e-05, -0.000329385698655038, -0.00062912867756193,
-0.000899442406382178, -0.000360003603924852, 0.000270014852457479,
-0.000720201687594013, -0.000240182539883449, 0, 0.000330236121826033,
3.00160585933185e-05, -0.000600492421815702, -0.000360468613094689,
0.000360468613094689, 0.000300291284800913, 0.000840336183904888,
-0.0005401350468901, 9.00427703767193e-05, -9.00427703767193e-05,
0.000180077433783232, -0.000330166737199988, 6.00384246096652e-05,
-9.00589886985159e-05, -0.000180142312913745, 0, 0.00048030740596694,
0, 6.00222082356439e-05, -0.000240110451961151, -6.0036622357984e-05,
0, 0.000300147074319135, 0.000899901071618636, -0.000359863255847692,
-0.000600060024006588, -0.000330186558403334, -0.000240204174702718,
-3.00295791371141e-05, 0.00066044252047881, -0.000150062276126128,
-0.000150084798193006, -0.000660640845624449, 9.01130919910287e-05,
-0.000210276514390095, -0.000270420505531277, 0, -6.01033778275806e-05,
9.01537121400509e-05, -6.01015716741315e-05, -0.00024044241520027,
0, -0.000300634340718808, 0.00012026458222536, -0.000210472511563253,
6.01395237129765e-05, 0.000601196398905657, 3.00503343124703e-05,
-0.000210371306129886, 0.000210371306129886, 0.000390572193220251,
0.000390419706138623, -0.000720893939666567, 0.000390548725916595,
-3.00367950760716e-05, 0.000450457221689682, 0.00203953161487558,
0.000329533712782215, 0.000898176761678471, -0.000299302627118969,
-0.000928407847341717, 0.00281252055423931, 0.000388343718939588,
-0.000537746853695786, -0.000119538581219381, 0.000209183140602143,
-0.000268957790190427, -0.000837220477081679, -5.98282928176985e-05,
-0.000149586393900947, 0.000239327490896279, -0.00101753764690216,
-0.000269521600793077, -0.000119810699238165, 0.000568973019077212,
0.00104726150712509, 0.000239220143477326, -0.000358851678492478,
-0.000179474141249081, 0.000926936541319812, 0.000388459761947679,
-8.96311678042494e-05, 0.000328608346621628, -0.000238977178817379,
0.000597335899728435, 0.000298534199308698, -0.000179109824652635,
0.000298498554498927, -0.000149238139576013, -0.000179115171533795,
-8.95696180451822e-05, 0.000298534199308698, 0.000895068234116003,
-0.000238606538951025, 0.00101368483084663, 0.00148884879957478,
-0.00122069237338707, -0.000953771941992976, -0.000208758928930131,
-0.000328137820828189, -2.98360509027873e-05, -2.98369411186883e-05,
0.000149175803961121, 0.000298284864254761, 0.00190691911667429,
-5.95362129187649e-05, -0.000804085991177317, -0.000774916586227192,
-0.000805357159753939, 0.000268524459715813, -0.000119335302507295,
-0.000746168459756724, 0.000776003620496013, 8.95001417680064e-05,
0.000298275967134387, 0.000745300912464053, -0.000178821566357179,
0.000506578076002739, -0.000596000852043588, -0.00014905573221391,
0.00014905573221391, -0.00131245358138621, 0.000208914688955453,
0, 8.95215076015532e-05, 0.000507137221315901, -0.000805573417872907,
8.95402110767662e-05, -2.98458462069107e-05, 0.00044759418874829,
-0.000298373864662693, -0.000119374477878686, -0.000746413517867417,
-0.000717081509110429, 0.000179318589840882, 0.00107523680500599,
0.000954796432374394, 0, 0.00140070654262203, 0.00344868934980624,
0.00216421424210278, -0.0024313597934924, -0.000415701650976708,
0.000326637272216601, 0.000148436224648663, -0.00118810718820583,
-0.000178337891330038, -0.00121949420981693, -0.00148920355146753,
-0.000357739092772924, -0.00170101057144478, -0.000119477881799668,
-0.000926437912742273, 8.96928022120491e-05, -0.00137617445281446,
-0.00143803014685062, 0.000958916497919127, 0.000239585518201579,
-0.000269537744403081, -5.99071439442156e-05, 0.00107777987799551,
0.00104672906753311, 0.000239098599423393, 0.000567630159388877,
2.98663480950978e-05, -5.97335882144279e-05, 0.000537473885736439,
-0.00098558956682826, -0.000268965828018608, 0.000179318589840882,
-0.000867004553764872, -0.00164638655421889, -0.000329602831575571,
2.99683833571152e-05, -0.0005395683584144, 0.000239844102484454,
-0.000119914860593262, -0.000119929241891192, 0, -0.000509859799603163,
0.000779680370562197, 0.000838976497943733, 0.000329405426190199,
0.000149693876302592, -5.98748615567857e-05, 5.98748615567857e-05,
-0.00026946511338366, 2.99441541544709e-05, 0.000209583976575445,
-0.000239528130729916, 0.000389204082574146, -2.99333981912397e-05,
-0.000299383272698961, -0.0016782047152466, -0.00168102582136953,
0.00039049006999381, -0.000811188435474897, -0.00249778188514682,
-0.00419705614818344, -0.00102930500732068, -0.00269938442070483,
0.000941062255446212, -0.000394531195842873, 0.00106184497940287,
-0.00154763561319715, -0.00082030720545756, -0.000395202850604015,
-0.00155192119394343, -0.00185938815706432, -0.00119060353233724,
0.0014041516348593, 0.00195032821230523, 0, -0.000548145454981608,
0.000213203381604465, 0.000334942073377142, -0.000213131975169922,
-0.000487329444341711, 0.000395973261748672, -0.00048737397778531,
0.000365552750285936, -0.000274152034249298, 0.00012185462757941,
0.000182754104861793, -9.13728775477907e-05, -0.000213235854893412,
-9.14007160366381e-05, 0.000456920062171129, -0.000975015311855287,
-0.000670874906996133, 0.000823283728713875, -0.000518284793845858,
0.000853502459908562, 0.00240417651066149, 0.000638113636262538,
-0.000911715607903929, -9.12172948623891e-05, 0, -0.000212872718123869,
0.0017624362640376, 0.000607017136522003, -0.000212414086886881,
-0.000607146128362324, 0.000455394151501309, -0.000364298728983137,
-9.10954225181726e-05, -0.0015498931853104, -0.000882384320385121,
9.13172514147575e-05, 0.00112556099798411, -0.000304080766194481,
0.000942349559803546, -0.000212711002519406, -0.000607995154767949,
-0.000121643402516192, 0.000395286968279507, -0.000182420723500165,
-0.000243279407598607, 0.000760051720103405, 6.07791892246468e-05,
0.000151931813294759, 0.0007289515570017, -0.000121455031424134,
6.07293596281977e-05, 0.00015180726578734, -0.000455490951197568,
0.000455490951197568, 0.000455283573645104, 0.000182055406031445,
-0.000394495278760232, 0.000273128690832181, -9.1034608385776e-05,
0.000273078967030216, -3.03384251338912e-05, 0.00230310111196452,
-0.00151460104078538, -0.000303195684815627, 0.000303195684815627,
-0.000879520869494499, -0.000819560215249204, -0.000273336068396191,
0.00166922146832427, 0.000485068980255576, 0.00015153581577998,
-0.00015153581577998, -0.00103105297828865, -0.000667718854479027,
0, 0.00036426555361313, 0.000333793566952956, 0.000212356091624955,
-0.000121340816165549, -0.000485510554219637, 0.000788834992361487,
0.000303232460326264, 0.000394064783286296, -0.000181856757994758,
-0.00012125621452963, -0.000212233761124558, -0.000151623125849376,
9.09766342305218e-05, 0.000303195684815627, 0.000363713518997777,
0, -0.000424345302196372, -0.000181917409998178, -0.000394267955671879,
-0.000394423464206817, -6.06943433059115e-05, 0.000333773310336127,
3.03375047430166e-05, 0.000545917761950676, -3.03209472285459e-05,
-0.000333591111374432, 0.00024262275313891, 0.000636604796035556,
0, -3.0305326164104e-05, -9.09214893560772e-05, -0.000394088675051307,
-0.000151613930578165, -0.00033363158284061, -0.000546182802751538,
-0.000819833953642046, -0.000546929611640934, 0.00100247589542768,
-0.00021256244101675, -0.000425260478447598, -0.000182309866507246,
0.000911217145332266, 0.000424950681763292, 0.000182066454758711,
-0.00048558422805467, 0.000151770402027296, 0.000151747371268662,
-0.000516035041123963, 0.000576727542584088, 0.0005763951196327,
-0.000333661942884866, 0.000363989326999103, 0.00112146711761341,
-0.00100016677782389, -0.000576307703592072, -0.000637358323167803,
-9.10843593606714e-05, -0.000182193611580672, -0.000425247561235409,
0.00106278002867644, 0.000151733556166711, -0.000151733556166711,
-0.00170099062943851, -0.000243235027043376, 0.000456017154147226,
-0.000303988329188876, -0.000395323029674266, 0, 0.000395323029674266,
-0.000212846827105651, -0.00100403142229322, -0.00048716622465772,
-0.000456933980991003, -0.000457142865104565, -0.000609849081257607,
0.000152497140973651, 0.000365898284360178, -0.000182932406985081,
-0.000579506825423515, -3.05096639383606e-05, -0.000579860544530852,
-0.000152650781105379, -0.00103864374695384, 3.05637483402066e-05,
-9.1694047592128e-05, 0.000122256861818215, -0.00122324174274357,
-0.000183615387485681, 0.00082600387105547, 0.000672536097739318,
0.000580462859038455, -0.000855536595816275, 0.000183390898212288,
0.000488878034669682, 0.00155670559343069, -9.15038661020162e-05,
0.000945136429141513, -0.000975639578594034, 0.00121940081567118,
-0.000609514540554024, -0.00176969595219223, -0.000641525018564515,
0.000794208429808485, 9.1598858132258e-05, -0.000152669425190766,
-0.000274863715139517, -0.0018343580306448, -0.00140862345682802,
-0.00104243325603637, -0.00110493857348359, -0.000430028265626881,
-0.00236846667984647, -0.00077019058321337, 0.000616199914743376,
0, 0.0019692919992389, -0.00153817787049437, -0.00181809827748936,
-0.000308480120902566, -0.000586374524941036, -0.00346352821730278,
-0.000123919576354048, -0.00130208351729788, 0.00111617535539477,
0.00130063191876495, 0.00111348253130217, -0.00105158985318621,
-0.0012075987285991, -0.00167447091746542, 0.000279273270709091,
-0.00528838412863397, -0.0039064380818159, 0.00084508369792502,
-0.00206702234573086, -0.00790020318445617, -0.00148628377427684,
0.00183381864973953, 0.000442142503413123, 0.000410385926365997,
0.000883336545543401, 0.00245661677513453, 0.00059749368905937,
-0.000157200572533434, -0.00144740593533221, -0.000314930875275898,
0.00427458533704783, 0.000438995334885561, -0.00160012584145353,
0.00116113050656796, 0.00219305203970244, -6.25919319201529e-05,
0.0014073717314611, 0.000312480472513421, -0.00193895482312723,
0.000281694551072853, -0.000751361878688961, -0.00144169016101348,
-0.000972747460484591, 0.00304061171835102, 0.00112612624513542,
-0.00234752913104153, -0.00106603132939931, 0.000313656611315771,
-0.000690174453296777, -0.000345265936667616, -0.00147657148142066,
-0.000566073353062357, -0.00182619698108066, -0.000693568754158491,
-0.00142016339965778, -0.000315865949118255, -0.000189567470790131,
0.00170481493539931, -0.00151524746460918, -0.00186564268567313,
-3.16510785127733e-05, 0.00208682530596072, -0.000600306490279756,
-0.00243651730379746, 0.000570107385662766, -0.00148929792883745,
0.00190084010980396, 0.000348095761493639, -0.000696312735840365,
-0.000886918018228045, -9.50736028864085e-05, -0.000253573807183827,
9.5097712971004e-05, 0.00234288535220895, 0.000948256858993801,
0.000568522803601645, -0.000284220999529516, 3.15841005669171e-05,
0.000252636898962599, 6.31492532807698e-05, 0.000599719096932461,
0.000630894945376603, -0.000567787536545517, -0.000189334175384026,
0.000599435286827266, -0.000378548900419773, -0.00113650725706993,
0.000284247929272397, 0.00028416715534707, 0.0014826734999005,
-0.000283746080691039, -0.000346910137984935, -0.00221043414493405,
-0.000664084141727805, 0.00135931865336136, 0.00309109506785976,
0.000566714957551007, 0.00113246732666816, -0.00169918228421917,
-0.0018913727543568, 0.00185987917117103, 0.000692629817067747,
3.14717776861428e-05, -0.00119662440277057, -0.000661907262594852,
-9.45939555165864e-05, 0.00028375502673228, -3.152436045184e-05,
-0.00053606623278224, 0.000126159086775601, -0.000599397465775198,
-0.00116827976218747, 0.000694817322587582, -0.00129527558507814,
0.000632051343607998, 0.000410619251177913, -0.00120074587056163,
0.000853309787409273, -0.000347556836939766, 0.000157995355264617,
-0.00034762273802702, -0.000885403548861241, 9.49021717486787e-05,
9.48931661817198e-05, 0.000822030461415046, 0.000537116330191267,
-9.47642739399512e-05, 0.0012312356530062, 0.000220830639571012,
-0.000946760571871152, 0.000536607705305059, 0.000157771011481955,
-0.000504954880387487, -0.000410463671835615, -0.000789827068940596,
-9.4821183123095e-05, 0.000347633723971619, 0.000284337731953244,
-0.000600363395866665, -0.000221277403738895, -9.48481639682086e-05,
0.00126390310050706, 0.000473552123259502, 0.000977964334928139,
-0.00145151645818764, -0.000631751868886532, -6.31971435094414e-05,
-3.16000695228524e-05, -9.48062003969241e-05, -0.000126422250484204,
0.000379218813796989, -0.000126390293393008, -0.000347655697944482,
3.161005832375e-05, 0.000347644710609885, -0.000474090939519556,
-0.000442687754265592, -0.000980935131264182, -0.00167931472744165,
0.000190252719602491, 0.00091904489189254, -0.000506954796827941,
-0.000158476094212823, 0, 0.00136207436443314, -0.000569944920753684,
0.000221683848072196, -0.000982116063671157, -0.000856178010475439,
-0.000507710869571909, -0.000507968770843803, 0.000317510718653935,
-0.000381024961744636, 0.000634960971234833, 0.00126871369466652,
-6.33974704618012e-05, 0.000316947167561032, 0.000285166586745156,
-0.000507019055763891, 0.000570378366435698, 0.00284702204199405,
0.00192504990415454, 0.00251905165010857, -3.1449013290441e-05,
-0.00141623023246318, -0.00110290074044972, -0.000662324784118873,
0.00258373650997967, 0.00110078481339215, 0.00191561915787464,
6.27450980594801e-05, 0, -0.000156870127542952, 0.000188241200279826,
0.0012540365941005, -0.000125332915721366, -0.000407440495818534,
-3.13484537404207e-05, -0.00100366975147459, -0.000251074915796679,
-6.27785799691694e-05, -6.27825213666e-05, 0.00100404765110262,
6.27195183353635e-05, -0.00050186632589444, -0.000784670713978919,
-0.00320785254796618, 0.000157487755652497, 0.000472314507261373,
0.00458557919014169, 0.00215986185728578, -0.00140807633133733,
6.26232896214773e-05, -3.13111546015321e-05, 0.00125168211103599,
6.25429983314163e-05, -9.38159644183756e-05, -0.000344067190332531,
-0.000625880164383474, 0.000187805183974987, 0.000500641457310103,
0.000750492545935622, 0, -9.37807718841555e-05, -6.25254009642973e-05,
0, -9.37954322308698e-05, -0.000781970919251584, -0.00187922881663383,
0.000250767978243083, 0.00228535749463532, -0.000719480753645918,
-0.000281676918438833, -0.000751526573652406, 0.000970615477675629,
3.12944968658968e-05, 0.000594409440454413, 0.00137517211320315,
0, 0.000374718965163723, 0.000593018010793145, 0, 0, -6.2406390434866e-05,
-0.000593055031037792, -0.000499687705588059, -0.00112521109577646,
0.000156355051368351, -0.000938497222109369, 0.00146992160823878,
-0.000218787604993054, 0.000874863358410138, -0.00281478887292064,
-0.000940026389958604, 0.000814740581544626, -9.3974658236462e-05,
-0.000908877285788812, 0.00072090147145687, -0.00128544796744468,
0.000564546495987805, 0.00115945675211293, 0.00156470067911751,
0.00162469572700186, -0.000999500333083603, -0.00134465364163461,
-0.00134646416985618, -0.00178764689841771, 0.00210093992394444,
-0.00137922409174784, -0.000282348514056174, -0.00474910864131139,
-0.000977779257512523, 0.000599397465775198, 0.0012922138047875,
-9.4495629647362e-05, -0.000913918409735537, -0.00119881394886079,
-0.00430230646857499, -0.000570812471894833, 0.00110961412668953,
0.000411829001750519, 0.00313068482440571, -0.000284203049201182,
-0.00237150511929318, -6.33171874726557e-05, -0.00101362061259991,
0.00344844291936575, 0.00324772865833545, -0.00176444686270738,
-0.00107279220209033, 0.00160875703040642, -0.00224034929544903,
-0.000631991425952272, 0.000947837422071274, -0.000536997564827502,
-0.00110649205921121, 3.16310553731824e-05, 0.000632411088270501,
9.48271775405019e-05, 0.00202084059931273, -0.000757336735471625,
-0.000505209988642896, -0.00132739187025965, -0.000727652404095558,
3.16480734268154e-05, 0.000885739650467343, 3.16190536437944e-05,
0.000442561807817654, 0.000821381230232454, 0.000662931126796273,
-0.000315626679386227, 0.000252509312621996, -0.0010420121247865,
-0.000126378313649589, 0.000252740657839645, -0.000979726071129505,
-0.00104399003455491, 0.000664483388633563, -0.000537881077972813,
0.000126586284545205, -0.00094978795210654, -0.000221747050880516,
-0.000348559028831907, -0.00107813303188031, 6.34517766711085e-05,
-0.000761687173834069, -0.00034930060847671, 0.000444543239150441,
0.00142755199118572, -0.000412194625111439, 0.000190264785733696,
0.000158526339482989, 0.000443740102368473, -3.16891924034834e-05,
-0.000951143028492751, -0.000571120363273003, -0.00019044596153428,
0.000380855660310431, -0.000380855660310431, 0.000507775320335746,
-0.000158652092160594, 0.000285555645630708, 0.000634276312016624,
0.000126806999916518, 0.0013305035700526, -3.16580926034149e-05,
-0.00022163471523573, -0.000126670466950252, -0.00063359312770217,
9.50645647552761e-05, 0.000475187311936409, -3.16721301114598e-05,
6.33432571310877e-05, 6.33392450177794e-05, 0.00486511353245334,
-6.30318310959055e-05, -9.4555196666235e-05, 9.4555196666235e-05,
-0.000409803772777551, -6.30616427770292e-05, 0.000283746080691039,
0.00141752398088357, -0.00018888714049492, -0.00015743320928685,
0.000849845033129348, -0.000440570230872517, 0, 0.000755144457254886,
0.000880281746984402, 0.00018853102962435, -0.000314238132240519,
-0.000188590288159318, 0, 0.000471409050355476, -0.000502844223173149,
-0.00028296103186598, -0.000251588151525262, -0.00132187725791599,
-0.000945268999395843, 0.00119718989756645, 0.000346287642566168,
-0.000188869303004324, 0.00147852237639245, 0.000471409050355476,
-0.000408541669076001, 9.42936619647838e-05, -0.000157161043244258,
-0.000377287308757523, 0.00150829591188462, 0, -3.13996389067484e-05,
0.000502275948402797, 0.000533391914502523, 0.000877908126868832,
-0.000156712792786173, -0.000595732690420014, 0.000376293513376424,
-0.000125415438804843, -0.000658689232290399, 0.000501897811595597,
0.000376258117504413, 0.000219418541747451, 0, 0.000438692702802612,
0.000501127547445357, -0.000407147001349806, 0.000469770290971283,
-0.000187881635122622, 0.000939055379364007, 0, 0, 0, -6.25762648427397e-05,
-0.000125164278278689, -0.00050081383292877, -0.000751691340831862,
-0.000814970424068306, 0.000532940433009443, 0.000156693148129605,
0.000532573123911639, -0.000469902738781336, -0.000470123651177268,
0.000470123651177268, 0.000125328988758611, 0.000720337023584428,
-0.000720337023584428, -0.000376034098188427, -0.000282118397557873,
0, 0.0006581524957463, 0.000501158940515722, -0.000344519789805986,
0.000845427695839618, 0.000187775796251621, 0.000250312892420723,
0.00184412511043153, 6.24531601500777e-05, -0.000343540658291452,
0.000218630437350598, 0.000561973164961316, 0.000842368039827512,
0.000405331675983334, 6.2344139671211e-05, -0.00084197401091668,
-0.00024960998569501, 0, -9.36198100198737e-05, -0.000718043207100116,
0.000593203158249622, 0.000374473401161701, -0.000530545381875136,
-0.000874453522306418, -0.000812677817989993, 0.00050018758076753,
-0.000437650449325311, 0.000218849166901514, -0.000187582067703751,
0.000250101605079323, -0.000468991832504706, 0.000312685659656253,
0.00103116953125859, -0.000624824288502523, 0.00137409846423342,
-6.24180762960336e-05, 0.000156037886315907, 0.000312002748154683,
0.000405458095223743, -0.000748666472876636, -0.000249680098672656,
-0.000499547295660818, -0.000124925825454092, 0.000312285306391935,
-0.000156130462931614, 0.000874016786815979, -3.12017348189642e-05,
-6.2406390434866e-05, -0.000280876962133192, -0.000374625379007121,
-0.000218596300347151, 0, -0.000437336005971289, -0.000625097691865584,
0.000562605503369618, -0.000156247558949829, 3.12514649145612e-05,
-0.00056267585104397, -0.000125082085281392, 0.000375199329042175,
0.000656260277620113, -0.000249953135089243, 9.37397472835499e-05,
9.37309609678749e-05, -0.000249968755207775, 0.00128042989964605,
-9.36344200805905e-05, 6.24239208661947e-05, 0, 0.00134117263526345,
0.000342801941746629, -0.00134071264918845, 0.000249571051053721,
-0.000218371263209782, 0.000218371263209782, 0.000873008504921557,
0.00326701079331926, -0.000310684440315789, -0.000372948786023564,
-0.000684100900462781, 0.000684100900462781, -0.00037308792871471,
-0.00136918119472718, -0.00037373863646728, 0.000622820149679626,
-0.000373645539283096, -0.000685379633670102, -0.000311691552566273,
-0.000187061574980163, -0.00184130503766777, 0.00224656089994024,
0.000186985789625282, -0.000997661813070039, -0.000780116432928857,
-0.000687006241305355, -0.00165700120762935, -0.000845189513617228,
0.00015657048052109, 0.00100147099410108, -0.000250273738205742,
-0.00162830785938084, 0.000689223084926915, 0.00125191716141337,
-0.00103271848825326, 0.0039061939326297, 0.00180730449293787,
-0.00105905816028784, -0.000436408984482384, -0.000654970783648956,
-0.000156008674398578, -0.000124824465756568, -0.000999157044437204,
-0.00143803951254107, 0.00125058637527609, -0.000249992189045756,
0.000312480472513421, 0.000374847722503624, 0, 0.00084289398575077,
0.000187213330135272, -9.36022839637118e-05, 0.00140311514530822,
-0.0007480830721045, -0.000686234781601236, 6.24044432164794e-05,
-0.000749110466394143, -0.000937177913709064, 0.000250000001301487,
-0.000625117229833272, -3.12661215957633e-05, -0.00103233081290455
), class = c("xts", "zoo"), .indexCLASS =
c("POSIXct", "POSIXt"
), tclass = c("POSIXct", "POSIXt"), .indexTZ = "",
tzone = "", index structure(c(1323734400,
1323734700, 1323735000, 1323735300, 1323735600, 1323735900, 1323736200,
1323736500, 1323736800, 1323737100, 1323737400, 1323737700, 1323738000,
1323738300, 1323738600, 1323738900, 1323739200, 1323739500, 1323739800,
1323740100, 1323740400, 1323740700, 1323741000, 1323741300, 1323741600,
1323741900, 1323742200, 1323742500, 1323742800, 1323743100, 1323743400,
1323743700, 1323744000, 1323744300, 1323744600, 1323744900, 1323745200,
1323745500, 1323745800, 1323746100, 1323746400, 1323746700, 1323747000,
1323747300, 1323747600, 1323747900, 1323748200, 1323748500, 1323748800,
1323749100, 1323749400, 1323749700, 1323750000, 1323750300, 1323750600,
1323750900, 1323751200, 1323751500, 1323751800, 1323752100, 1323752400,
1323752700, 1323753000, 1323753300, 1323753600, 1323753900, 1323754200,
1323754500, 1323754800, 1323755100, 1323755400, 1323755700, 1323756000,
1323756300, 1323756600, 1323756900, 1323757200, 1323757500, 1323757800,
1323758100, 1323758400, 1323758700, 1323759000, 1323759300, 1323759600,
1323759900, 1323760200, 1323760500, 1323760800, 1323761100, 1323761400,
1323761700, 1323762000, 1323762300, 1323762600, 1323762900, 1323763200,
1323763500, 1323763800, 1323764100, 1323764400, 1323764700, 1323765000,
1323765300, 1323765600, 1323765900, 1323766200, 1323766500, 1323766800,
1323767100, 1323767400, 1323767700, 1323768000, 1323768300, 1323768600,
1323768900, 1323769200, 1323769500, 1323769800, 1323770100, 1323770400,
1323770700, 1323771000, 1323771300, 1323771600, 1323771900, 1323772200,
1323772500, 1323772800, 1323773100, 1323773400, 1323773700, 1323774000,
1323774300, 1323774600, 1323774900, 1323775200, 1323775500, 1323775800,
1323776100, 1323776400, 1323776700, 1323777000, 1323777300, 1323777600,
1323777900, 1323778200, 1323778500, 1323778800, 1323779100, 1323779400,
1323779700, 1323780000, 1323780300, 1323780600, 1323780900, 1323781200,
1323781500, 1323781800, 1323782100, 1323782400, 1323782700, 1323783000,
1323783300, 1323783600, 1323783900, 1323784200, 1323784500, 1323784800,
1323785100, 1323785400, 1323785700, 1323786000, 1323786300, 1323786600,
1323786900, 1323787200, 1323787500, 1323787800, 1323788100, 1323788400,
1323788700, 1323789000, 1323789300, 1323789600, 1323789900, 1323790200,
1323790500, 1323790800, 1323791100, 1323791400, 1323791700, 1323792000,
1323792300, 1323792600, 1323792900, 1323793200, 1323793500, 1323793800,
1323794100, 1323794400, 1323794700, 1323795000, 1323795300, 1323795600,
1323795900, 1323796200, 1323796500, 1323796800, 1323797100, 1323797400,
1323797700, 1323798000, 1323798300, 1323798600, 1323798900, 1323799200,
1323799500, 1323799800, 1323800100, 1323800400, 1323800700, 1323801000,
1323801300, 1323801600, 1323801900, 1323802200, 1323802500, 1323802800,
1323803100, 1323803400, 1323803700, 1323804000, 1323804300, 1323804600,
1323804900, 1323805200, 1323805500, 1323805800, 1323806100, 1323806400,
1323806700, 1323807000, 1323807300, 1323807600, 1323807900, 1323808200,
1323808500, 1323808800, 1323809100, 1323809400, 1323809700, 1323810000,
1323810300, 1323810600, 1323810900, 1323811200, 1323811500, 1323811800,
1323812100, 1323812400, 1323812700, 1323813000, 1323813300, 1323813600,
1323820800, 1323821100, 1323821400, 1323821700, 1323822000, 1323822300,
1323822600, 1323822900, 1323823200, 1323823500, 1323823800, 1323824100,
1323824400, 1323824700, 1323825000, 1323825300, 1323825600, 1323825900,
1323826200, 1323826500, 1323826800, 1323827100, 1323827400, 1323827700,
1323828000, 1323828300, 1323828600, 1323828900, 1323829200, 1323829500,
1323829800, 1323830100, 1323830400, 1323830700, 1323831000, 1323831300,
1323831600, 1323831900, 1323832200, 1323832500, 1323832800, 1323833100,
1323833400, 1323833700, 1323834000, 1323834300, 1323834600, 1323834900,
1323835200, 1323835500, 1323835800, 1323836100, 1323836400, 1323836700,
1323837000, 1323837300, 1323837600, 1323837900, 1323838200, 1323838500,
1323838800, 1323839100, 1323839400, 1323839700, 1323840000, 1323840300,
1323840600, 1323840900, 1323841200, 1323841500, 1323841800, 1323842100,
1323842400, 1323842700, 1323843000, 1323843300, 1323843600, 1323843900,
1323844200, 1323844500, 1323844800, 1323845100, 1323845400, 1323845700,
1323846000, 1323846300, 1323846600, 1323846900, 1323847200, 1323847500,
1323847800, 1323848100, 1323848400, 1323848700, 1323849000, 1323849300,
1323849600, 1323849900, 1323850200, 1323850500, 1323850800, 1323851100,
1323851400, 1323851700, 1323852000, 1323852300, 1323852600, 1323852900,
1323853200, 1323853500, 1323853800, 1323854100, 1323854400, 1323854700,
1323855000, 1323855300, 1323855600, 1323855900, 1323856200, 1323856500,
1323856800, 1323857100, 1323857400, 1323857700, 1323858000, 1323858300,
1323858600, 1323858900, 1323859200, 1323859500, 1323859800, 1323860100,
1323860400, 1323860700, 1323861000, 1323861300, 1323861600, 1323861900,
1323862200, 1323862500, 1323862800, 1323863100, 1323863400, 1323863700,
1323864000, 1323864300, 1323864600, 1323864900, 1323865200, 1323865500,
1323865800, 1323866100, 1323866400, 1323866700, 1323867000, 1323867300,
1323867600, 1323867900, 1323868200, 1323868500, 1323868800, 1323869100,
1323869400, 1323869700, 1323870000, 1323870300, 1323870600, 1323870900,
1323871200, 1323871500, 1323871800, 1323872100, 1323872400, 1323872700,
1323873000, 1323873300, 1323873600, 1323873900, 1323874200, 1323874500,
1323874800, 1323875100, 1323875400, 1323875700, 1323876000, 1323876300,
1323876600, 1323876900, 1323877200, 1323877500, 1323877800, 1323878100,
1323878400, 1323878700, 1323879000, 1323879300, 1323879600, 1323879900,
1323880200, 1323880500, 1323880800, 1323881100, 1323881400, 1323881700,
1323882000, 1323882300, 1323882600, 1323882900, 1323883200, 1323883500,
1323883800, 1323884100, 1323884400, 1323884700, 1323885000, 1323885300,
1323885600, 1323885900, 1323886200, 1323886500, 1323886800, 1323887100,
1323887400, 1323887700, 1323888000, 1323888300, 1323888600, 1323888900,
1323889200, 1323889500, 1323889800, 1323890100, 1323890400, 1323890700,
1323891000, 1323891300, 1323891600, 1323891900, 1323892200, 1323892500,
1323892800, 1323893100, 1323893400, 1323893700, 1323894000, 1323894300,
1323894600, 1323894900, 1323895200, 1323895500, 1323895800, 1323896100,
1323896400, 1323896700, 1323897000, 1323897300, 1323897600, 1323897900,
1323898200, 1323898500, 1323898800, 1323899100, 1323899400, 1323899700,
1323900000, 1323907200, 1323907500, 1323907800, 1323908100, 1323908400,
1323908700, 1323909000, 1323909300, 1323909600, 1323909900, 1323910200,
1323910500, 1323910800, 1323911100, 1323911400, 1323911700, 1323912000,
1323912300, 1323912600, 1323912900, 1323913200, 1323913500, 1323913800,
1323914100, 1323914400, 1323914700, 1323915000, 1323915300, 1323915600,
1323915900, 1323916200, 1323916500, 1323916800, 1323917100, 1323917400,
1323917700, 1323918000, 1323918300, 1323918600, 1323918900, 1323919200,
1323919500, 1323919800, 1323920100, 1323920400, 1323920700, 1323921000,
1323921300, 1323921600, 1323921900, 1323922200, 1323922500, 1323922800,
1323923100, 1323923400, 1323923700, 1323924000, 1323924300, 1323924600,
1323924900, 1323925200, 1323925500, 1323925800, 1323926100, 1323926400,
1323926700, 1323927000, 1323927300, 1323927600, 1323927900, 1323928200,
1323928500, 1323928800, 1323929100, 1323929400, 1323929700, 1323930000,
1323930300, 1323930600, 1323930900, 1323931200, 1323931500, 1323931800,
1323932100, 1323932400, 1323932700, 1323933000, 1323933300, 1323933600,
1323933900, 1323934200, 1323934500, 1323934800, 1323935100, 1323935400,
1323935700, 1323936000, 1323936300, 1323936600, 1323936900, 1323937200,
1323937500, 1323937800, 1323938100, 1323938400, 1323938700, 1323939000,
1323939300, 1323939600, 1323939900, 1323940200, 1323940500, 1323940800,
1323941100, 1323941400, 1323941700, 1323942000, 1323942300, 1323942600,
1323942900, 1323943200, 1323943500, 1323943800, 1323944100, 1323944400,
1323944700, 1323945000, 1323945300, 1323945600, 1323945900, 1323946200,
1323946500, 1323946800, 1323947100, 1323947400, 1323947700, 1323948000,
1323948300, 1323948600, 1323948900, 1323949200, 1323949500, 1323949800,
1323950100, 1323950400, 1323950700, 1323951000, 1323951300, 1323951600,
1323951900, 1323952200, 1323952500, 1323952800, 1323953100, 1323953400,
1323953700, 1323954000, 1323954300, 1323954600, 1323954900, 1323955200,
1323955500, 1323955800, 1323956100, 1323956400, 1323956700, 1323957000,
1323957300, 1323957600, 1323957900, 1323958200, 1323958500, 1323958800,
1323959100, 1323959400, 1323959700, 1323960000, 1323960300, 1323960600,
1323960900, 1323961200, 1323961500, 1323961800, 1323962100, 1323962400,
1323962700, 1323963000, 1323963300, 1323963600, 1323963900, 1323964200,
1323964500, 1323964800, 1323965100, 1323965400, 1323965700, 1323966000,
1323966300, 1323966600, 1323966900, 1323967200, 1323967500, 1323967800,
1323968100, 1323968400, 1323968700, 1323969000, 1323969300, 1323969600,
1323969900, 1323970200, 1323970500, 1323970800, 1323971100, 1323971400,
1323971700, 1323972000, 1323972300, 1323972600, 1323972900, 1323973200,
1323973500, 1323973800, 1323974100, 1323974400, 1323974700, 1323975000,
1323975300, 1323975600, 1323975900, 1323976200, 1323976500, 1323976800,
1323977100, 1323977400, 1323977700, 1323978000, 1323978300, 1323978600,
1323978900, 1323979200, 1323979500, 1323979800, 1323980100, 1323980400,
1323980700, 1323981000, 1323981300, 1323981600, 1323981900, 1323982200,
1323982500, 1323982800, 1323983100, 1323983400, 1323983700, 1323984000,
1323984300, 1323984600, 1323984900, 1323985200, 1323985500, 1323985800,
1323986100, 1323986400, 1323993600, 1323993900, 1323994200, 1323994500,
1323994800, 1323995100, 1323995400, 1323995700, 1323996000, 1323996300,
1323996600, 1323996900, 1323997200, 1323997500, 1323997800, 1323998100,
1323998400, 1323998700, 1323999000, 1323999300, 1323999600, 1323999900,
1324000200, 1324000500, 1324000800, 1324001100, 1324001400, 1324001700,
1324002000, 1324002300, 1324002600, 1324002900, 1324003200, 1324003500,
1324003800, 1324004100, 1324004400, 1324004700, 1324005000, 1324005300,
1324005600, 1324005900, 1324006200, 1324006500, 1324006800, 1324007100,
1324007400, 1324007700, 1324008000, 1324008300, 1324008600, 1324008900,
1324009200, 1324009500, 1324009800, 1324010100, 1324010400, 1324010700,
1324011000, 1324011300, 1324011600, 1324011900, 1324012200, 1324012500,
1324012800, 1324013100, 1324013400, 1324013700, 1324014000, 1324014300,
1324014600, 1324014900, 1324015200, 1324015500, 1324015800, 1324016100,
1324016400, 1324016700, 1324017000, 1324017300, 1324017600, 1324017900,
1324018200, 1324018500, 1324018800, 1324019100, 1324019400, 1324019700,
1324020000, 1324020300, 1324020600, 1324020900, 1324021200, 1324021500,
1324021800, 1324022100, 1324022400, 1324022700, 1324023000, 1324023300,
1324023600, 1324023900, 1324024200, 1324024500, 1324024800, 1324025100,
1324025400, 1324025700, 1324026000, 1324026300, 1324026600, 1324026900,
1324027200, 1324027500, 1324027800, 1324028100, 1324028400, 1324028700,
1324029000, 1324029300, 1324029600, 1324029900, 1324030200, 1324030500,
1324030800, 1324031100, 1324031400, 1324031700, 1324032000, 1324032300,
1324032600, 1324032900, 1324033200, 1324033500, 1324033800, 1324034100,
1324034400, 1324034700, 1324035000, 1324035300, 1324035600, 1324035900,
1324036200, 1324036500, 1324036800, 1324037100, 1324037400, 1324037700,
1324038000, 1324038300, 1324038600, 1324038900, 1324039200, 1324039500,
1324039800, 1324040100, 1324040400, 1324040700, 1324041000, 1324041300,
1324041600, 1324041900, 1324042200, 1324042500, 1324042800, 1324043100,
1324043400, 1324043700, 1324044000, 1324044300, 1324044600, 1324044900,
1324045200, 1324045500, 1324045800, 1324046100, 1324046400, 1324046700,
1324047000, 1324047300, 1324047600, 1324047900, 1324048200, 1324048500,
1324048800, 1324049100, 1324049400, 1324049700, 1324050000, 1324050300,
1324050600, 1324050900, 1324051200, 1324051500, 1324051800, 1324052100,
1324052400, 1324052700, 1324053000, 1324053300, 1324053600, 1324053900,
1324054200, 1324054500, 1324054800), tzone = "", tclass =
c("POSIXct",
"POSIXt")), .Dim = c(1000L, 1L))
The full code is just below this code and what I have tried to do within it
is change the following two lines when running transform="log" but
this
gives unrealistic regression output such as values over 3:
if( type == "HARRVJ" ){
J = J[(maxp:(n-h)),];
*x = cbind(x1,J);* # bind jumps to RV data
if(!is.null(transform)){ y = Ftransform(y); x = Ftransform(x); }
x = cbind(x,rmin);
model = estimhar(y=y,x=x);
model$transform = transform; model$h = h; model$type = "HARRVJ";
model$dates = alldates[(maxp+h):n];
class(model) = c("harModel","lm");
return( model )
}#End HAR-RV-J if cond
to
if( type == "HARRVJ" ){
J = J[(maxp:(n-h)),];
*x = cbind(x1,J+1); * # bind jumps to RV data
if(!is.null(transform)){ y = Ftransform(y); x = Ftransform(x); }
x = cbind(x,rmin);
model = estimhar(y=y,x=x);
model$transform = transform; model$h = h; model$type = "HARRVJ";
model$dates = alldates[(maxp+h):n];
class(model) = c("harModel","lm");
return( model )
}#End HAR-RV-J if cond
and this
if( type == "HARRVCJ" ){
# Are the jumps significant? if not set to zero:
if( jumptest=="ABDJumptest" ){
TQ = apply.daily(data, TQfun);
J = J[,1];
teststats = ABDJumptest(RV=RM1,BPV=RM2,TQ=TQ );
}else{ jtest = match.fun(jumptest); teststats = jtest(data,...) }
Jindicators = teststats > qnorm(1-alpha);
*J[!Jindicators] = 0;*
to
if( type == "HARRVCJ" ){
# Are the jumps significant? if not set to zero:
if( jumptest=="ABDJumptest" ){
TQ = apply.daily(data, TQfun);
J = J[,1];
teststats = ABDJumptest(RV=RM1,BPV=RM2,TQ=TQ );
}else{ jtest = match.fun(jumptest); teststats = jtest(data,...) }
Jindicators = teststats > qnorm(1-alpha);
* J[!Jindicators] = 1;*
the full code is (which I take no credit for):
# START implementation of paper:
# ROUGHING IT UP: INCLUDING JUMP COMPONENTS IN THE MEASUREMENT, MODELING,
AND FORECASTING OF RETURN VOLATILITY
# Torben G. Andersen, Tim Bollerslev, and Francis X. Diebold
# data: a xts object with the intraday data
# periods: a vector with time periods to aggregate over, expressed in days
# RVest: estimator for daily realized volatility,
# in case a vector is supplied, the first estimator is the unrobust
estimator, the second is the robust estimator
# type: string defining the type of model
# "HARRV" from "roughing paper"
# "HARRVJ" from "roughing paper"
# "HARRVCJ" from "roughing paper"
# jumptest: function to calculate the jump test statistic which determines
whether the daily jump contribution is significant
# alpha: a value between zero and one to indicate what
# h: integer, determining over how many periods the depend variable should
be aggregated. The default is 1, i.e. no aggregation is done, just one day.
# TODO ADD extra argument: jump-periods??? for aggregated jumps in the
model...
# Helpfunctions:
TQfun = function(rdata){ #Calculate the realized tripower quarticity
returns = as.vector(as.numeric(rdata));
n = length(returns);
mu43 = 0.8308609; # 2^(2/3)*gamma(7/6) *gamma(1/2)^(-1)
tq = n * ((mu43)^(-3)) * sum( abs(returns[1:(n - 2)])^(4/3)
*abs(returns[2:(n-1)])^(4/3) *abs(returns[3:n])^(4/3) );
return(tq);
}
ABDJumptest = function(RV, BPV, TQ){ # Comput jump detection stat mentioned
in roughing paper
mu1 = sqrt(2/pi);
n = length(RV);
zstat = ((1/n)^(-1/2))*((RV-BPV)/RV)*( (mu1^(-4) + 2*(mu1^(-2))-5) *
pmax( 1,TQ*(BPV^(-2)) ) )^(-1/2);
return(zstat);
}
harModel = function(data, periods = c(1,5,22), periodsJ = c(1,5,22),
leverage=NULL, RVest = c("RCov","RBPCov"),
type="HARRV",
jumptest="ABDJumptest",alpha=0.001,h=1,transform=NULL,
...){
nperiods = length(periods); # Number of periods to aggregate over
nest = length(RVest); # Number of RV estimators
if( !is.null(transform) ){ Ftransform = match.fun(transform); }
if( !(type %in% c("HARRV","HARRVJ","HARRVCJ"))){
warning("Please provide a
valid argument for type, see documentation.") }
if( sum(data<0) != 0 ){ #If it are returns as input
# Get the daily RMs (in a non-robust and robust way)
RV1 = match.fun( RVest[1]);
RM1 = apply.daily( data, RV1 );
# save dates:
alldates = index(RM1)
if( nest == 2 ){
RV2 = match.fun( RVest[2]);
RM2 = apply.daily( data, RV2 ); }
}
if( sum(data<0) == 0 ){ #The input is most likely already realized
measures
dimdata = dim(data)[2];
alldates = index(data);
RM1 = data[,1];
if( dimdata > 1 ){ RM2 = data[,2]; }
if( type != "HARRV" ){ warning("Please provide returns as
input for the
type of model you want to estimate. All your returns are positive which is
quite unlikely honestly. Only for the HAR-RV model you can input realized
measures.") }
}
# Get the matrix for estimation of linear model
maxp = max(periods,periodsJ); #max number of aggregation levels
if(!is.null(leverage)){ maxp = max(maxp,leverage) }
n = length(RM1); #Number of Days
# Aggregate RV:
RVmatrix1 = aggRV(RM1,periods);
if( nest==2 ){ RVmatrix2 = aggRV(RM2,periods); } # In case a jumprobust
estimator is supplied
# Aggregate and subselect y:
y = aggY(RM1,h,maxp);
# Only keep useful parts:
x1 = RVmatrix1[(maxp:(n-h)),];
if( nest==2 ){ x2 = RVmatrix2[(maxp:(n-h)),]; } # In case a jumprobust
estimator is supplied
# Jumps:
if(type!="HARRV"){ # If model type is as such that you need jump
component
J = pmax( RM1 - RM2,0 ); # Jump contributions should be positive
J = aggJ(J,periodsJ);
}
if( !is.null(leverage) ){
if( sum(data<0) == 0 ){ warning("You cannot use leverage variables
in
the model in case your input consists of Realized Measures") }
# Get close-to-close returns
e = apply.daily(data,sum); #Sum logreturns daily
# Get the rmins:
rmintemp = pmin(e,0);
# Aggregate everything:
rmin = aggRV(rmintemp,periods=leverage,type="Rmin");
# Select:
rmin = rmin[(maxp:(n-h)),];
}else{ rmin = matrix(ncol=0,nrow=dim(x1)[1]) }
###############################
# Estimate the model parameters, according to type of model :
# First model type: traditional HAR-RV:
if( type == "HARRV" ){
if(!is.null(transform)){ y = Ftransform(y); x1 = Ftransform(x1) }
x1 = cbind(x1,rmin);
model = estimhar(y=y,x=x1);
model$transform = transform; model$h = h; model$type = "HARRV";
model$dates = alldates[(maxp+h):n];
class(model) = c("harModel","lm");
return( model )
} #End HAR-RV if cond
if( type == "HARRVJ" ){
J = J[(maxp:(n-h)),];
x = cbind(x1,J); # bind jumps to RV data
if(!is.null(transform)){ y = Ftransform(y); x = Ftransform(x); }
x = cbind(x,rmin);
model = estimhar(y=y,x=x);
model$transform = transform; model$h = h; model$type = "HARRVJ";
model$dates = alldates[(maxp+h):n];
class(model) = c("harModel","lm");
return( model )
}#End HAR-RV-J if cond
if( type == "HARRVCJ" ){
# Are the jumps significant? if not set to zero:
if( jumptest=="ABDJumptest" ){
TQ = apply.daily(data, TQfun);
J = J[,1];
teststats = ABDJumptest(RV=RM1,BPV=RM2,TQ=TQ );
}else{ jtest = match.fun(jumptest); teststats = jtest(data,...) }
Jindicators = teststats > qnorm(1-alpha);
J[!Jindicators] = 0;
# Get continuus components if necessary RV measures if necessary:
Cmatrix = matrix( nrow = dim(RVmatrix1)[1], ncol = 1 );
Cmatrix[Jindicators,] = RVmatrix2[Jindicators,1]; #Fill with
robust one in case of jump
Cmatrix[(!Jindicators)] = RVmatrix1[(!Jindicators),1]; #Fill with
non-robust one in case of no-jump
# Aggregate again:
Cmatrix <- aggRV(Cmatrix,periods,type="C");
Jmatrix <- aggJ(J,periodsJ);
# subset again:
Cmatrix <- Cmatrix[(maxp:(n-h)),];
Jmatrix <- Jmatrix[(maxp:(n-h)),];
x = cbind(Cmatrix,Jmatrix); # bind jumps to RV data
if(!is.null(transform)){ y = Ftransform(y); x = Ftransform(x); }
x = cbind(x,rmin);
model = estimhar( y=y, x=x );
model$transform = transform; model$h = h; model$type = "HARRVCJ";
model$dates = alldates[(maxp+h):n];
class(model) = c("harModel","lm");
return(model)
}
} #End function harModel
#################################################################
estimhar = function(y, x){ #Potentially add stuff here
colnames(y)="y";
output = lm( formula(y~x), data=cbind(y,x));
}
# Help function to get nicely formatted formula's for print/summary
methods..
getHarmodelformula = function(x){
modelnames = colnames(x$model$x);
if(!is.null(x$transform)){
modelnames =
paste(x$transform,"(",modelnames,")",sep=""); }
#Added
visual tingie for plotting transformed RV
betas =
paste("beta",(1:length(modelnames)),"",sep="")
betas2 = paste(" + ",betas,"*")
rightside = paste(betas2, modelnames,collapse="");
h = x$h;
left = paste("RV",h,sep="");
if(!is.null(x$transform)){ left =
paste(x$transform,"(",left,")",sep="" )
}
modeldescription = paste(left,"= beta0",rightside);
return(list(modeldescription,betas))
}
aggRV <- function(RM1,periods,type="RV"){
n = length(RM1);
nperiods = length(periods);
RVmatrix1 = matrix(nrow=n,ncol=nperiods);
for(i in 1:nperiods){
if(periods[i]==1){ RVmatrix1[,i] = RM1;
}else{ RVmatrix1[(periods[i]:n),i]
rollmean(x=RM1,k=periods[i],align="left") }
} #end loop over periods for standard RV estimator
colnames(RVmatrix1) = paste(type,periods,sep="");
return(RVmatrix1);
}
aggJ <- function( J, periodsJ ){
n = length(J);
nperiods = length(periodsJ);
JM = matrix(nrow=n,ncol=nperiods);
for(i in 1:nperiods){
if(periodsJ[i]==1){ JM[,i] = J;
}else{ JM[(periodsJ[i]:n),i] = rollmean( x=J, k=periodsJ[i],
align="left") }
} # End loop over periods for standard RV estimator
colnames(JM) = paste("J",periodsJ,sep="");
return(JM)
}
aggY = function(RM1,h,maxp){
n = length(RM1);
if( h == 1 ){ y = RM1[(maxp+1):n]; }
if( h != 1 ){
y = matrix( nrow=length(RM1), ncol=1 ); colnames(y) = "y";
y[(h:n),] = rollmean(x=RM1,k=h,align="left");
y = matrix(y[((maxp+h):n),],ncol=1); y=as.data.frame(y) }
return(y);
}
#########################################################################
# Print method for harmodel:
print.harModel = function(x, digits = max(3, getOption("digits") - 3),
...){
formula = getHarmodelformula(x); modeldescription = formula[[1]]; betas
formula[[2]];
cat("\nModel:\n", paste(modeldescription, sep = "\n",
collapse = "\n"),
"\n\n", sep = "")
coefs = coef(x);
names(coefs) = c("beta0",betas)
if (length(coef(x))){
cat("Coefficients:\n")
print.default(format(coefs, digits = digits), print.gap = 2,quote FALSE);
cat("\n\n");
Rs = summary(x)[c("r.squared", "adj.r.squared")]
zz = c(Rs$r.squared,Rs$adj.r.squared);
names(zz) = c("r.squared","adj.r.squared")
print.default((format(zz,digits=digits) ),print.gap = 2,quote=FALSE)
}
else cat("No coefficients\n")
cat("\n")
invisible(x)
}
summary.harModel = function(object, correlation = FALSE, symbolic.cor
FALSE,...){
x=object;
dd = summary.lm(x);
formula = getHarmodelformula(x); modeldescription = formula[[1]]; betas
formula[[2]];
dd$call = modeldescription;
rownames(dd$coefficients) = c("beta0",betas);
return(dd)
}
plot.harModel = function(x, which = c(1L:3L, 5L), caption = list("Residuals
vs Fitted",
"Normal
Q-Q", "Scale-Location", "Cook's distance",
"Residuals vs Leverage",
expression("Cook's dist vs Leverage " * h[ii]/(1 - h[ii]))),
panel = if (add.smooth) panel.smooth else points,
sub.caption = NULL,
main = "", ask = prod(par("mfcol"))
< length(which)
&& dev.interactive(),
..., id.n = 3, labels.id = names(residuals(x)),
cex.id = 0.75,
qqline = TRUE, cook.levels = c(0.5, 1), add.smooth
= getOption("add.smooth"),
label.pos = c(4, 2), cex.caption = 1){
observed = x$model$y;
fitted = x$fitted.values;
dates = x$dates;
dates = as.POSIXct(dates);
observed = xts(observed, order.by=dates);
fitted = xts(fitted, order.by=dates);
type = x$type;
g_range = range(fitted,observed)
g_range[1] = 0.95*g_range[1]; g_range[2]= 1.05 * g_range[2];
#ind = seq(1,length(fitted),length.out=5);
title = paste("Observed and forecasted RV based on HAR
Model:",type);
plot.zoo(observed,col="red",lwd=2,main=title,
ylim=g_range,xlab="Time",ylab="Realized Volatility");
# axis(1,time(b)[ind], format(time(b)[ind],), las=2, cex.axis=0.8); not
used anymore
# axis(2);
lines(fitted,col="blue",lwd=2);
legend("topleft", c("Observed RV","Forecasted
RV"), cex=1.1,
col=c("red","blue"),lty=1, lwd=2, bty="n");
}
If anyone could help me with as to how J should be modified when the model
is in log form it would be hugely appreciated, Thanks in advance.
--
View this message in context:
http://r.789695.n4.nabble.com/Switching-log-J-to-log-J-1-to-avoid-log-0-in-HAR-RVJ-model-tp4637088.html
Sent from the R help mailing list archive at Nabble.com.
Joshua Ulrich
2012-Jul-19 20:20 UTC
[R] Switching log(J) to log(J+1) to avoid log(0) in HAR-RVJ model
Cross-posted on Stack Overflow: http://stackoverflow.com/q/11567745/271616 -- Joshua Ulrich | FOSS Trading: www.fosstrading.com On Thu, Jul 19, 2012 at 12:23 PM, cursethiscure <caolan.harvey6 at mail.dcu.ie> wrote:> I am working with xts dependent data, and my code is as follows (the problem > is explained throughout): > > dat <- getdat("prices") > dat <- read.zoo(dat, sep = "",format="%d/%m/%Y %H:%M", > tz="", FUN=NULL, regular=TRUE, > header=TRUE, index.column=1, colClasses=c("character", > "numeric")) > dat <- as.xts(dat) > ## cleaned data here to get > daylist <- lapply(split(dat, "days"), function(x) { > if(NROW(x) >= 10) x > }) > do.call(rbind, daylist) -> dat > makeReturns(dat) -> dat > > the code I am running to get the HAR regressions (full code for it shown > below) is > > x = harModel(dat, periods = c(1,5,22), periodsJ=c(1), RVest > c("RCov","RBPCov"), > type="HARRVJ", h=5, transform="log") ; # Estimate the HAR model > of type HARRVJ > > The three HAR models on paper are: > > 1.?RV?_(t,t+h) = ?_0+ ?_D ?RV?_t+ ?_W ?RV?_(t-5)+ ?_M ?RV?_(t-22) + ?_J J_t > + ?_(t,t+h) # NULL model in code at bottom > > 2. (RV?_(t,t+h) )^(1/2) = ?_0+ ?_D ??RV?_t?^(1/2)+ ?_W (?RV?_(t-5) )^(1/2)+ > ?_M (?RV?_(t-22) )^(1/2) + ?_J (? J?_t )^(1/2) + ?_(t,t+h) > > 3. log(RV?_(t,t+h) )= ?_0+ ?_D.log(?RV?_t )+ ?_W.log(?RV?_(t-5) )+ > ?_M.log(?RV?_(t-22) ) + ?_J log(J_t + 1) + ?_(t,t+h) > > Basically the harModel in the code allows you to transform the regressions > from NULL to "sqrt" or "log", but when `transform="log"` is chosen it gives > the following error message. > > x = harModel(dat, periods = c(1,5,22), periodsJ=c(1), RVest > c("RCov","RBPCov"), > + type="HARRVJ", h=22, transform="log") ; # Estimate .... > [TRUNCATED] > Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) : > NA/NaN/Inf in foreign function call (arg 1) > > which is due to it taking the log(0), the actual log model should take > log(J + 1) in case of a 0 value for the J in the time series ( if J does not > occur it should return a 0, otherwise it should return J), but unfortunately > I do not know how to rectify this. I was wondering if any one could tell me > how how I can achieve this as I am very naive with R still. I have tried > some modifications you will see at the bottom of the question, but they do > not compute the regression correctly though they do allow it to run without > 'error' > > A sample of the xts returns are ><snip>> > The full code is just below this code and what I have tried to do within it > is change the following two lines when running transform="log" but this > gives unrealistic regression output such as values over 3: > > if( type == "HARRVJ" ){ > J = J[(maxp:(n-h)),]; > *x = cbind(x1,J);* # bind jumps to RV data > if(!is.null(transform)){ y = Ftransform(y); x = Ftransform(x); } > x = cbind(x,rmin); > model = estimhar(y=y,x=x); > model$transform = transform; model$h = h; model$type = "HARRVJ"; > model$dates = alldates[(maxp+h):n]; > class(model) = c("harModel","lm"); > return( model ) > }#End HAR-RV-J if cond > > to > > if( type == "HARRVJ" ){ > J = J[(maxp:(n-h)),]; > *x = cbind(x1,J+1); * # bind jumps to RV data > if(!is.null(transform)){ y = Ftransform(y); x = Ftransform(x); } > x = cbind(x,rmin); > model = estimhar(y=y,x=x); > model$transform = transform; model$h = h; model$type = "HARRVJ"; > model$dates = alldates[(maxp+h):n]; > class(model) = c("harModel","lm"); > return( model ) > }#End HAR-RV-J if cond > > and this > > if( type == "HARRVCJ" ){ > # Are the jumps significant? if not set to zero: > if( jumptest=="ABDJumptest" ){ > TQ = apply.daily(data, TQfun); > J = J[,1]; > teststats = ABDJumptest(RV=RM1,BPV=RM2,TQ=TQ ); > }else{ jtest = match.fun(jumptest); teststats = jtest(data,...) } > Jindicators = teststats > qnorm(1-alpha); > *J[!Jindicators] = 0;* > to > if( type == "HARRVCJ" ){ > # Are the jumps significant? if not set to zero: > if( jumptest=="ABDJumptest" ){ > TQ = apply.daily(data, TQfun); > J = J[,1]; > teststats = ABDJumptest(RV=RM1,BPV=RM2,TQ=TQ ); > }else{ jtest = match.fun(jumptest); teststats = jtest(data,...) } > Jindicators = teststats > qnorm(1-alpha); > * J[!Jindicators] = 1;* > > the full code is (which I take no credit for): ><snip>> > If anyone could help me with as to how J should be modified when the model > is in log form it would be hugely appreciated, Thanks in advance. > > -- > View this message in context: http://r.789695.n4.nabble.com/Switching-log-J-to-log-J-1-to-avoid-log-0-in-HAR-RVJ-model-tp4637088.html > Sent from the R help mailing list archive at Nabble.com. > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.