Displaying 8 results from an estimated 8 matches for "out1_miss".
2024 Dec 11
1
Cores hang when calling mcapply
...dcast(dt[!is.na(column2)], ID_Key ~ column2, fun.aggregate =
> length, value.var = "column2")
>
> # Step D: Merge the two wide tables by ID_Key
> # Fill missing columns with 0 using data.table on-the-fly operations
> all_cols <- unique(c(names(out1), names(out2)))
> out1_missing <- setdiff(all_cols, names(out1))
> out2_missing <- setdiff(all_cols, names(out2))
>
> # Add missing columns with 0
> for (col in out1_missing) out1[, (col) := 0]
> for (col in out2_missing) out2[, (col) := 0]
>
> # Ensure column order alignment if needed
> setcolor...
2024 Dec 12
1
Cores hang when calling mcapply
...t;column2")
out1 <- setDT(out1 |> rename_with(~ paste0("column1_name_", .x, recycle0 = TRUE), -ID_Key))
out2 <- setDT(out1 |> rename_with(~ paste0("column2_name_", .x, recycle0 = TRUE), -ID_Key))
all_cols <- unique(c(names(out1), names(out2)))
out1_missing <- setdiff(all_cols, names(out1))
out2_missing <- setdiff(all_cols, names(out2))
for (col in out1_missing) out1[, (col) := 0]
for (col in out2_missing) out2[, (col) := 0]
setcolorder(out1, all_cols)
setcolorder(out2, all_cols)
final_dt <- rbindlist(list(out1, out...
2024 Dec 11
1
Cores hang when calling mcapply
...dcast(dt[!is.na(column2)], ID_Key ~ column2, fun.aggregate =
> length, value.var = "column2")
>
> # Step D: Merge the two wide tables by ID_Key
> # Fill missing columns with 0 using data.table on-the-fly operations
> all_cols <- unique(c(names(out1), names(out2)))
> out1_missing <- setdiff(all_cols, names(out1))
> out2_missing <- setdiff(all_cols, names(out2))
>
> # Add missing columns with 0
> for (col in out1_missing) out1[, (col) := 0]
> for (col in out2_missing) out2[, (col) := 0]
>
> # Ensure column order alignment if needed
> setcolor...
2024 Dec 11
1
Cores hang when calling mcapply
...> > > length, value.var = "column2")
> > >
> > > # Step D: Merge the two wide tables by ID_Key
> > > # Fill missing columns with 0 using data.table on-the-fly operations
> > > all_cols <- unique(c(names(out1), names(out2)))
> > > out1_missing <- setdiff(all_cols, names(out1))
> > > out2_missing <- setdiff(all_cols, names(out2))
> > >
> > > # Add missing columns with 0
> > > for (col in out1_missing) out1[, (col) := 0]
> > > for (col in out2_missing) out2[, (col) := 0]
> > >...
2024 Dec 12
1
Cores hang when calling mcapply
...; ? ? out1 <- setDT(out1 |> rename_with(~ paste0("column1_name_", .x, recycle0 = TRUE), -ID_Key))
> ? ? out2 <- setDT(out1 |> rename_with(~ paste0("column2_name_", .x, recycle0 = TRUE), -ID_Key))
> ? ? all_cols <- unique(c(names(out1), names(out2)))
> ? ? out1_missing <- setdiff(all_cols, names(out1))
> ? ? out2_missing <- setdiff(all_cols, names(out2))
> ? ? for (col in out1_missing) out1[, (col) := 0]
> ? ? for (col in out2_missing) out2[, (col) := 0]
> ? ? setcolorder(out1, all_cols)
> ? ? setcolorder(out2, all_cols)
> ? ? final_dt...
2024 Dec 11
1
Cores hang when calling mcapply
...column2, fun.aggregate =
>
> > length, value.var = "column2")
> >
> > # Step D: Merge the two wide tables by ID_Key
> > # Fill missing columns with 0 using data.table on-the-fly operations
> > all_cols <- unique(c(names(out1), names(out2)))
> > out1_missing <- setdiff(all_cols, names(out1))
> > out2_missing <- setdiff(all_cols, names(out2))
> >
> > # Add missing columns with 0
> > for (col in out1_missing) out1[, (col) := 0]
> > for (col in out2_missing) out2[, (col) := 0]
> >
> > # Ensure column orde...
2024 Dec 11
1
Cores hang when calling mcapply
...dcast(dt[!is.na(column2)], ID_Key ~ column2, fun.aggregate =
> length, value.var = "column2")
>
> # Step D: Merge the two wide tables by ID_Key
> # Fill missing columns with 0 using data.table on-the-fly operations
> all_cols <- unique(c(names(out1), names(out2)))
> out1_missing <- setdiff(all_cols, names(out1))
> out2_missing <- setdiff(all_cols, names(out2))
>
> # Add missing columns with 0
> for (col in out1_missing) out1[, (col) := 0]
> for (col in out2_missing) out2[, (col) := 0]
>
> # Ensure column order alignment if needed
> setcolor...
2024 Dec 11
2
Cores hang when calling mcapply
Hi R users.
Apologies for the lack of concrete examples because the dataset is large, and it being so I believe is the issue.
I multiple, very large datasets for which I need to generate 0/1 absence/presence columns
Some include over 200M rows, with two columns that need presence/absence columns based on the strings contained within them, as an example, one set has ~29k unique values and the