R help - Mar 2023 - loess plotting problem

For some reason the following code is not plotting as I want it to. I want
to plot a "loess" line plotted over a scatter plot. I get a jumble,
with
lines connecting all the points. I had a similar problem with
"lowess". I
solved that by dropping "NA" rows from the data columns. Please help.

library(stats)
attach(gini_pci_wdi_narm)
plot(ny_gnp_pcap_pp_kd, si_pov_gini)
lines(loess(si_pov_gini ~ ny_gnp_pcap_pp_kd, gini_pci_wdi_narm))
detach(gini_pci_wdi_narm)

-- 
Anupam.

	[[alternative HTML version deleted]]

? Thu, 23 Mar 2023 19:48:40 +0530
Anupam Tyagi <anuptyagi at gmail.com> ?????:
> lines(loess(si_pov_gini ~ ny_gnp_pcap_pp_kd, gini_pci_wdi_narm))
lines() on an object of class "loess" ends up calling
lines.default(...), which uses xy.coords(...) to obtain the points to
plot, which, in turn, ends up extracting the $x and $y components from
the list returned by loess(). The structure of the "loess" object is
_not_ documented, but it turns out that the $x and the $y components
are the source data points which were originally used to fit the model,
not the smooth predictions.

Solution: use predict() on the loess object in order to obtain the
smooth predictions.

-- 
Best regards,
Ivan

Dear Anupam Tyagi,

You didn't include your data, so it's not possible to see exactly what 
happened, but I think that you misunderstand the object that loess() 
returns. It returns a "loess" object with several components,
including
the original data in x and y. So if pass the object to lines(), you'll 
simply connect the points, and if x isn't sorted, the points won't be in
order. Try, e.g.,

plot(speed ~ dist, data=cars)
m <- loess(speed ~ dist, data=cars)
names(m)
lines(m)

You'd do better to use loess.smooth(), which is intended for adding a 
loess regression to a scatterplot; for example,

plot(speed ~ dist, data=cars)
with(cars, lines(loess.smooth(dist, speed)))

Other points: You don't have to load the stats package which is 
available by default when you start R. It's best to avoid attach(), the 
use of which can cause confusion.

I hope this helps,
  John

-- 
* preferred email: john.david.fox at proton.me
John Fox, Professor Emeritus
McMaster University
Hamilton, Ontario, Canada
web: https://www.john-fox.ca/

On 2023-03-23 10:18 a.m., Anupam Tyagi wrote:
> For some reason the following code is not plotting as I want it to. I want
> to plot a "loess" line plotted over a scatter plot. I get a
jumble, with
> lines connecting all the points. I had a similar problem with
"lowess". I
> solved that by dropping "NA" rows from the data columns. Please
help.
> 
> library(stats)
> attach(gini_pci_wdi_narm)
> plot(ny_gnp_pcap_pp_kd, si_pov_gini)
> lines(loess(si_pov_gini ~ ny_gnp_pcap_pp_kd, gini_pci_wdi_narm))
> detach(gini_pci_wdi_narm)
>