Fedora directory users - Jun 2005 - schema extension via ldif at install time

Hi all, 

I want to perform a custom installation of FDS, and when it asks for the ldif 
file to be imported, I want to point it at a file that contains ldif to 
extend the schema, and add the entries dumped from my old openldap directory. 

Without extending the schema, the import will fail because every entry 
violates the default schema. However, turning schema checking off leaves some 
small chance that there is something in my directory that shouldn''t be
there
or isn''t as it should be. 

I have another installation of FDS on a test machine, and I used the GUI to 
add my extra objects and attributes, and I tried copying the resulting ldif 
to the server I''m now trying to install (minus operational stuff), but
the
install ignored the cn=schema part of my LDIF file (the part holding the 
schema extensions), and subsequently, all of the entries failed. 

Has anyone done this? Any clues? Is there somewhere I can place a 99user.ldif 
file *before* running setup that will cause the slapd instance to use it the 
first time it starts up? 
Thanks, 
brian.

Brian K. Jones wrote:
>I want to perform a custom installation of FDS, and when it asks for the
ldif
>file to be imported, I want to point it at a file that contains ldif to 
>extend the schema, and add the entries dumped from my old openldap
directory.
>
>Without extending the schema, the import will fail because every entry 
>violates the default schema. However, turning schema checking off leaves
some
>small chance that there is something in my directory that shouldn''t
be there
>or isn''t as it should be. 
>
>I have another installation of FDS on a test machine, and I used the GUI to 
>add my extra objects and attributes, and I tried copying the resulting ldif 
>to the server I''m now trying to install (minus operational stuff),
but the
>install ignored the cn=schema part of my LDIF file (the part holding the 
>schema extensions), and subsequently, all of the entries failed. 
>
>Has anyone done this? Any clues? Is there somewhere I can place a
99user.ldif
>file *before* running setup that will cause the slapd instance to use it the
>first time it starts up? 
>  
>
I''m not sure if there''s a ''clean'' way to do
what you want (there may be,
I''m not terribly familiar with the details of installation), however I 
suspect
that the reason it''s not working for you is that the ldif file is 
_imported_,
and not added via LDAP. You can''t import schema entries because they
don''t live in an importable back end. They need to be added via an
LDAP connection.

Actually, I figured it out, and just performed an install that worked. 

I put my 99user.ldif in /opt/fedora-ds/bin/slapd/install/schema before running 
setup. Then, when I ran setup, I selected "custom install". Then I
just fed
it my ldif dump from openldap, and there were no errors in the ldif import 
stage of the installation. If I find that things really didn''t happen
after I
fire up the console, I''ll check back, but all appears well at the
moment.

brian.

On Monday 20 June 2005 1:26 pm, David Boreham wrote:
> Brian K. Jones wrote:
> >I want to perform a custom installation of FDS, and when it asks for
the
> > ldif file to be imported, I want to point it at a file that contains
ldif
> > to extend the schema, and add the entries dumped from my old openldap
> > directory.
> >
> >Without extending the schema, the import will fail because every entry
> >violates the default schema. However, turning schema checking off
leaves
> > some small chance that there is something in my directory that
shouldn''t
> > be there or isn''t as it should be.
> >
> >I have another installation of FDS on a test machine, and I used the
GUI
> > to add my extra objects and attributes, and I tried copying the
resulting
> > ldif to the server I''m now trying to install (minus
operational stuff),
> > but the install ignored the cn=schema part of my LDIF file (the part
> > holding the schema extensions), and subsequently, all of the entries
> > failed.
> >
> >Has anyone done this? Any clues? Is there somewhere I can place a
> > 99user.ldif file *before* running setup that will cause the slapd
> > instance to use it the first time it starts up?
>
> I''m not sure if there''s a ''clean'' way
to do what you want (there may be,
> I''m not terribly familiar with the details of installation),
however I
> suspect
> that the reason it''s not working for you is that the ldif file is
> _imported_,
> and not added via LDAP. You can''t import schema entries because
they
> don''t live in an importable back end. They need to be added via an
> LDAP connection.
>
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

Brian K. Jones wrote:
> Actually, I figured it out, and just performed an install that worked. 
> 
> I put my 99user.ldif in /opt/fedora-ds/bin/slapd/install/schema before
running
> setup. Then, when I ran setup, I selected "custom install". Then
I just fed
> it my ldif dump from openldap, and there were no errors in the ldif import 
> stage of the installation. If I find that things really didn''t
happen after I
> fire up the console, I''ll check back, but all appears well at the
moment.
> 
Don''t put schema into 99user.ldif, it''s not maintainable.

Instead, create a file like 90myOwnSchema.ldif, then you can version 
control and upgrade it when needed.

Also, you should read about the silent installation capability of the 
setup program. I have done extensive coding to generate silent.inf files 
from arbitrary data sources, and have automated the entire installatoin 
and replicaton configuration process from start to finish.

--
mike

On Monday 20 June 2005 2:03 pm, Mike Jackson wrote:
> Don''t put schema into 99user.ldif, it''s not maintainable.
This would seem to be contrary to the documentation on redhat''s site.
I''m
certainly not meaning to say you''re wrong - quite the contrary -
I''m pointing
out to whomever it may concern that the docs need polishing :)
>
> Instead, create a file like 90myOwnSchema.ldif, then you can version
> control and upgrade it when needed.
I''ve also seen the docs mention that these files are imported in order,
so I
understand, and will change this. Presumably, I can do a simple "mv"
on the
file. Lemme know if this is not the case. 
>
> Also, you should read about the silent installation capability of the
I did. Thanks!
> setup program. I have done extensive coding to generate silent.inf files
Seen ''em, thanks!
> from arbitrary data sources, and have automated the entire installatoin
> and replicaton configuration process from start to finish.
Right - and I plan to use this in the future to make disaster recovery, moving 
of services/machines as simple and brainless as humanly possible. The first 
step, though, was to get the import right before I go automating it ;-) 

Did any of your coding allow for a "userSchema" keyword in the config
file,
for which I could''ve just put the path to my schema file? That would be
nice,
but I don''t remember seeing it. I did see that I could specify an ldif
file,
but attempts to put schema extensions in that ldif file have, so far, failed. 

thanks, 
brian.
>
> --
> mike
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

Brian K. Jones wrote:
>On Monday 20 June 2005 2:03 pm, Mike Jackson wrote:
>
>  
>
>>Don''t put schema into 99user.ldif, it''s not
maintainable.
>>    
>>
>
>This would seem to be contrary to the documentation on redhat''s
site. I''m
>certainly not meaning to say you''re wrong - quite the contrary -
I''m pointing
>out to whomever it may concern that the docs need polishing :)
>  
>
99user.ldif is for schema that is added over LDAP.  However, whatever is 
in the file at start up will remain there through other restarts.  But 
in general, it''s best to keep user defined schema in files named 
60x.ldif through 98x.ldif.
>  
>
>>Instead, create a file like 90myOwnSchema.ldif, then you can version
>>control and upgrade it when needed.
>>    
>>
>
>I''ve also seen the docs mention that these files are imported in
order, so I
>understand, and will change this. Presumably, I can do a simple
"mv" on the
>file. Lemme know if this is not the case. 
>  
>
Sure.  Just make sure there is nothing else in 99user.ldif that was 
added over LDAP.
>  
>
>>Also, you should read about the silent installation capability of the
>>    
>>
>
>I did. Thanks!
>
>  
>
>>setup program. I have done extensive coding to generate silent.inf files
>>    
>>
>
>Seen ''em, thanks!
>
>  
>
>>from arbitrary data sources, and have automated the entire installatoin
>>and replicaton configuration process from start to finish.
>>    
>>
>
>Right - and I plan to use this in the future to make disaster recovery,
moving
>of services/machines as simple and brainless as humanly possible. The first 
>step, though, was to get the import right before I go automating it ;-) 
>
>Did any of your coding allow for a "userSchema" keyword in the
config file,
>for which I could''ve just put the path to my schema file? That
would be nice,
>but I don''t remember seeing it. I did see that I could specify an
ldif file,
>but attempts to put schema extensions in that ldif file have, so far,
failed.
>  
>
No.  But that would be a nice enhancement.
>thanks, 
>brian.
>
>  
>
>>--
>>mike
>>
>>--
>>Fedora-directory-users mailing list
>>Fedora-directory-users@redhat.com
>>https://www.redhat.com/mailman/listinfo/fedora-directory-users
>>    
>>
>
>--
>Fedora-directory-users mailing list
>Fedora-directory-users@redhat.com
>https://www.redhat.com/mailman/listinfo/fedora-directory-users
>  
>

Brian K. Jones wrote:
> On Monday 20 June 2005 2:03 pm, Mike Jackson wrote:
> 
> 
>>Don''t put schema into 99user.ldif, it''s not
maintainable.
> 
> 
> This would seem to be contrary to the documentation on redhat''s
site. I''m
> certainly not meaning to say you''re wrong - quite the contrary -
I''m pointing
> out to whomever it may concern that the docs need polishing :)
OK, I guess "not maintainable" is ambiguous in this context. Here
comes
the longer explanation.

Every piece of schema which you add over-the-wire gets mixed into 
99user.ldif.

Let''s say that you write version 1.0 of your custom schema and you 
deploy it into 99user.ldif. When you write version 2.0 of your cool, 
custom schema, and you have a pool of servers to update, which are not 
in the same namespace partitions, you have a hard time figuring out 
which machine has which version, etc.

It''s just not a clean solution.

> I''ve also seen the docs mention that these files are imported in
order, so I
> understand, and will change this. Presumably, I can do a simple
"mv" on the
> file. Lemme know if this is not the case. 
I think if you move 99user.ldif, the server will refuse to start, or it 
will create an empty 99user.ldif. You could just move it and then touch 
an empty one, but check the permissions and ownership...

> Right - and I plan to use this in the future to make disaster recovery,
moving
> of services/machines as simple and brainless as humanly possible. The first
> step, though, was to get the import right before I go automating it ;-) 
I have also been doing a lot of work on automated disaster recovery over 
  the past few months. That is really not fun.

> Did any of your coding allow for a "userSchema" keyword in the
config file,
> for which I could''ve just put the path to my schema file? That
would be nice,
> but I don''t remember seeing it. I did see that I could specify an
ldif file,
> but attempts to put schema extensions in that ldif file have, so far,
failed.
The silent.inf doesn''t allow you to specify schema files. What I do is 
to use a script to copy them somewhere, generate the silent.inf, run 
setup, symlink the files into place, restart the server, then import my 
data.


Mike
-- 
LDAP Directory Consulting - http://www.netauth.com

Brian K. Jones wrote:
>On Monday 20 June 2005 2:03 pm, Mike Jackson wrote:
>
>  
>
>>Don''t put schema into 99user.ldif, it''s not
maintainable.
>>    
>>
>This would seem to be contrary to the documentation on redhat''s
site. I''m
>certainly not meaning to say you''re wrong - quite the contrary -
I''m pointing
>out to whomever it may concern that the docs need polishing :)
>
Keep in mind that you can edit schema using the console if the schema 
you are editing is in 99user.ldif.  If you put it in any other file, the 
console will see it as read-only.

As a "good" practice, I''d say if you are testing or have
schema that
changes often (for whatever reason), use 99user.ldif, but once you have 
a stable schema, put it in a separate file for maintainability.  If you 
are rolling out multiple apps with custom schema in each, it would 
probably be userful to put separate apps customizations in separate 
files (i.e. 98mail.ldif, 98cal.ldif, etc).  Just makes it easier to keep 
things straight.

 - Jeff

Rich Megginson wrote:
>>
>>Did any of your coding allow for a "userSchema" keyword in the
config file,
>>for which I could''ve just put the path to my schema file? That
would be nice,
>>but I don''t remember seeing it. I did see that I could specify
an ldif file,
>>but attempts to put schema extensions in that ldif file have, so far,
failed.
>>  
>>
> No.  But that would be a nice enhancement.

Indeed, especially if it were able to support multiple lines, e.g. 
multiple schema files.

  And while we''re at it, let''s fix setup to be able to parse
tabs and
arbitrary amounts of whitespace in silent.inf, in front of and behind 
the equals signs. The current limitation of the parser forces the file 
to be quite messy and unreadable (for a neat-freak like myself).

--
mike

Mike Jackson wrote:
> Rich Megginson wrote:
>
>>>
>>> Did any of your coding allow for a "userSchema" keyword
in the
>>> config file, for which I could''ve just put the path to my
schema
>>> file? That would be nice, but I don''t remember seeing it.
I did see
>>> that I could specify an ldif file, but attempts to put schema 
>>> extensions in that ldif file have, so far, failed.  
>>>
>> No.  But that would be a nice enhancement.
>
>
>
> Indeed, especially if it were able to support multiple lines, e.g. 
> multiple schema files.
>
>  And while we''re at it, let''s fix setup to be able to
parse tabs and
> arbitrary amounts of whitespace in silent.inf, in front of and behind 
> the equals signs. The current limitation of the parser forces the file 
> to be quite messy and unreadable (for a neat-freak like myself).
Once we open source it, we would be glad to have people help us enhance 
it ;-)
>
> -- 
> mike
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

As I understand it, replicated schema is stored in the 99user.ldif file 
on each consumer.
Does anyone know if schema defined in 60x.ldif through 98x.ldif files on 
the master is replicated to consumers?  Or will masters only replicate 
the schema that is present in their local 99user.ldif file?

In other words, will schema changes made to "60custom.ldif" on the 
master be replicated to each consumer''s 99user.ldif file, or will each 
consumer''s /<serverroot>/config/schema directory need to get an
updated
copy of 60custom.ldif ?

- George


Jeff Clowser wrote:
>
>
> Brian K. Jones wrote:
>
>> On Monday 20 June 2005 2:03 pm, Mike Jackson wrote:
>>
>>  
>>
>>> Don''t put schema into 99user.ldif, it''s not
maintainable.
>>>   
>>
>> This would seem to be contrary to the documentation on
redhat''s site.
>> I''m certainly not meaning to say you''re wrong - quite
the contrary -
>> I''m pointing out to whomever it may concern that the docs need
>> polishing :)
>>
> Keep in mind that you can edit schema using the console if the schema 
> you are editing is in 99user.ldif.  If you put it in any other file, 
> the console will see it as read-only.
>
> As a "good" practice, I''d say if you are testing or have
schema that
> changes often (for whatever reason), use 99user.ldif, but once you 
> have a stable schema, put it in a separate file for maintainability.  
> If you are rolling out multiple apps with custom schema in each, it 
> would probably be userful to put separate apps customizations in 
> separate files (i.e. 98mail.ldif, 98cal.ldif, etc).  Just makes it 
> easier to keep things straight.
>
> - Jeff
>
>
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>

George Holbert wrote:
> As I understand it, replicated schema is stored in the 99user.ldif file 
> on each consumer.
> Does anyone know if schema defined in 60x.ldif through 98x.ldif files on 
> the master is replicated to consumers?  Or will masters only replicate 
> the schema that is present in their local 99user.ldif file?
Custom schema files are not replicated. You must put the same files on 
all LDAP servers.


Mike

-- 
LDAP Directory Consulting - http://www.netauth.com

George Holbert wrote:
> As I understand it, replicated schema is stored in the 99user.ldif 
> file on each consumer.
> Does anyone know if schema defined in 60x.ldif through 98x.ldif files 
> on the master is replicated to consumers?
It is not.
> Or will masters only replicate the schema that is present in their 
> local 99user.ldif file?
Only schema added over LDAP will be automatically replicated.
>
> In other words, will schema changes made to "60custom.ldif" on
the
> master be replicated to each consumer''s 99user.ldif file, or will
each
> consumer''s /<serverroot>/config/schema directory need to get
an
> updated copy of 60custom.ldif ?
No.  Because there is no way to update the schema in the fie 
60custom.ldif via LDAP.
>
> - George
>
>
> Jeff Clowser wrote:
>
>>
>>
>> Brian K. Jones wrote:
>>
>>> On Monday 20 June 2005 2:03 pm, Mike Jackson wrote:
>>>
>>>  
>>>
>>>> Don''t put schema into 99user.ldif, it''s not
maintainable.
>>>>   
>>>
>>>
>>> This would seem to be contrary to the documentation on
redhat''s
>>> site. I''m certainly not meaning to say you''re
wrong - quite the
>>> contrary - I''m pointing out to whomever it may concern
that the docs
>>> need polishing :)
>>>
>> Keep in mind that you can edit schema using the console if the schema 
>> you are editing is in 99user.ldif.  If you put it in any other file, 
>> the console will see it as read-only.
>>
>> As a "good" practice, I''d say if you are testing or
have schema that
>> changes often (for whatever reason), use 99user.ldif, but once you 
>> have a stable schema, put it in a separate file for maintainability.  
>> If you are rolling out multiple apps with custom schema in each, it 
>> would probably be userful to put separate apps customizations in 
>> separate files (i.e. 98mail.ldif, 98cal.ldif, etc).  Just makes it 
>> easier to keep things straight.
>>
>> - Jeff
>>
>>
>>
>> -- 
>> Fedora-directory-users mailing list
>> Fedora-directory-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>>
>
>
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users

Rich Megginson wrote:
Mike
Jackson wrote:

  [...].

 And while we''re at it, let''s fix setup to be able to parse
tabs and
arbitrary amounts of whitespace in silent.inf, in front of and behind
the equals signs. The current limitation of the parser forces the file
to be quite messy and unreadable (for a neat-freak like myself).

Once we open source it, we would be glad to have people help us enhance
it ;-)

Thank you for pointing it up, Mike.  Not to forget it, I''ve filed a bug
in bugzilla:

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=161135

--noriko

-- 

mike

-- 

Fedora-directory-users mailing list

Fedora-directory-users@redhat.com

https://www.redhat.com/mailman/listinfo/fedora-directory-users

--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users

Noriko Hosoi wrote:
> 
> Thank you for pointing it up, Mike.  Not to forget it, I''ve filed
a bug
> in bugzilla:
> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=161135
>

Mike Jackson wrote:
> [...]
>  And while we''re at it, let''s fix setup to be able to
parse tabs and
> arbitrary amounts of whitespace in silent.inf, in front of and behind 
> the equals signs. The current limitation of the parser forces the file 
> to be quite messy and unreadable (for a neat-freak like myself).
>
> -- 
> mike
>
> -- 
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
Hello; could it be possible to share your silent.inf that causes the 
error?  Attached is my test silent.inf (sorry,  very ugly... :), with 
which my DS has been installed just fine.  I''m running the test on 
RHEL4.  Do you think there could be some requirements to duplicate the 
problem?
Thank you,
--noriko