configure error

[barronh]

Hi Carlie,

I downloaded the latest version via git clone and was following the install instructions. I noticed that the source directory Makefile.template has a sed command that would be very useful to quickly apply changes to other directories. However, the configure recipe is circular -- but easily fixed.

Current recipe:

configure: ${IODIR}/Makefile ${TOOLDIR}/Makefile
	(cd $(IODIR)   ;  sed $(SEDCMD) < Makefile.$(CPLMODE).sed > Makefile )
	(cd $(TOOLDIR) ;  sed $(SEDCMD) < Makefile.$(CPLMODE).sed > Makefile )

Note that the recipe depends on the files it will produce. So, if the product files have ever been made, it will work. If they have not been made (the initial case), it will fail. The recipe actually depends on the $(CPLMODE).sed version as revised below.

configure: ${IODIR}/Makefile.$(CPLMODE).sed ${TOOLDIR}/Makefile.$(CPLMODE).sed
	(cd $(IODIR)   ;  sed $(SEDCMD) < Makefile.$(CPLMODE).sed > Makefile )
	(cd $(TOOLDIR) ;  sed $(SEDCMD) < Makefile.$(CPLMODE).sed > Makefile )

I think it would be great if the default configuration was even easier. For example, modify Makefile.template to automatically select your recommended "starting point" for users. Instead of having all options commented out, you could have a cascading set of common pairings. The code below can be inserted into Makefile.template. It defaults to nocpl and chooses sane options, but allows for any variable to be overwritten. With cpl, it again defaults to sane options and allows for overwriting options.

ifndef CPLMODE
CPLMODE    = nocpl
endif

ifndef SYSNAME
  SYSNAME = $(shell uname -s)
endif
ifndef SYSVERSION
  SYSVERSION = $(shell uname -r | cut -d. -f1)
endif
ifndef SYSARCH
  SYSARCH = $(shell uname -p)
endif
ifndef BIN
  ifeq ($(SYSNAME),Linux)
  ifneq ($(SYSVERSION),2)
    $(warning Linux version is $(SYSVERSION); newer versions use IOAPI config for version 2.)
    SYSVERSION = 2
  endif
  endif
  BIN        = $(SYSNAME)$(SYSVERSION)_$(SYSARCH)
endif
export BIN

ifndef BASEDIR
  BASEDIR    = ${PWD}
endif

ifndef INSTALL
  INSTALL    = ${HOME}
endif

ifndef LIBINST
  LIBINST    = $(INSTALL)/$(BIN)
endif

ifndef BININST
  BININST    = $(INSTALL)/$(BIN)
endif

ifndef NCFLIBS
  NCFLIBS    = $(shell nc-config --libs) $(shell nf-config --libs)
ifeq ($(CPLMOD),pncf)
  NCFLIBS = $(NCFLIBS) -lpnetcdf
endif
endif

ifndef IOAPIDEFS
ifeq ($(CPLMOD),nocpl)
  IOAPIDEFS  = 
endif
ifeq ($(CPLMOD),cpl)
  IOAPIDEFS  = "-DIOAPICPL"
endif
ifeq ($(CPLMOD),pncf)
  IOAPIDEFS  = "-DIOAPI_PNCF"
endif
endif

ifndef (PVMINCL)
PVMINCL   = 
ifeq ($(CPLMOD),cpl)
#  turn on PVM coupling mode
PVMINCL   = $(PVM_ROOT)/conf/$(PVM_ARCH).def
endif
endif

If you then moved Makefile.template to Makefile, the standard installation procedure would be simplified to

git clone https://github.com/cjcoats/ioapi-3.2
cd ioapi-3.2
make all

Of course, a more savvy user could modify the environment and/or the Makefile script. A less savvy user would be helped quite a bit.

Thanks for considering.

[cjcoats]

First of all, the recommended procedure has never been to use the top-level ioapi-3.2/Makefile;
it has always been to set up and use make in ioapi-3.2/ioapi, ioapi-3.2/m3tools, and ioapi-3.2/tests.  The top-level make thing has not been fully tested (as you have done), and has been used instead for multi-BIN updates, mostly.

Good catch on the Makefile.$(CPLMODE).sed dependencies for make configure.

There is good reason for the Makefile.templates:  they keep a tar-ball download from CMAS from clobbering whatever has already been done to the _Makefile_s.

For make-variables: note that there are serious problems with variable-scoping for GNU make: instead of following the pre-existing POSIX Standard, it does its own thing" and  breaks commands like

make BIN=Linux2_x86_64ifort_medium

that are needed for multi-type builds.  That is why it is better to have the user control variables like BIN, rather than letting make botch it.  And I really do want to encourage multiple builds (particularly optimized and debug, small and medium model builds) from the same source tree.  I find it really obnoxious that CMAQ discourages such.  (Note also that the standard scripts for both SMOKE and CMAQ do NOT support debugger use, although they claim to -- I've been complaining about that one for decades!) (I do doubt that most of your users want to do test-builds for all of the compilers at hand (GNU, Intel, PG, Sun, g95), the way I do....)

There are serious problems with the usage of nc-config and nf-config that you suggest:
they may well not be in the PATH at all; they may well point to different library-versions than you want:  in particular, both run-time-library usage and Fortran linking are frequently dependent upon the underlying compiler-set. That's one of the reasons why the instructions ask you to do your own netCDF build/installs per $BIN for yourself. And do all the client-software build systems adjust themselves appropriately for, e.g., HDF5 support or not?

-- Carlie

[barronh]

Carlie,

I really do want to encourage multiple builds

Great, I think the approach I am proposing will encourage multiple builds. The current approach requires the builder to have multiple files that they copy and/or edit between builds. This actually creates a small barrier to multiple builds. The proposed approach allows the builder to make many different builds without edits or multiple files. That will increase the ease of making multiple builds. With the current top-level file and my proposed edits, you can make 5 builds in 5 lines (excluding comments) as shown below.

# Default build
make configure && make all
# Default build in a new INSTALL path
make INSTALL=/somewhere/else/ configure && make all
# Build with alternative cpl
make CPLMODE=cpl configure && make all
# Build with alternative Makeinclude....
make BIN=Linux2_x86_64intel configure && make all
# Build with alternative NCFLIBS
make NCFLIBS='...' INSTALLPATH=~/ioapi_altncf configure && make all

And, of course, it does not stop the user from making edits or having multiple files. So, they have more options.

There are serious problems with the usage of nc-config and nf-config that you suggest ...
they may well point to different library-versions than you want

I agree that nc-config may not always be "right." However, it is not clear to me that it is better to assume they are always wrong. As you'll note from the ifndef sections, the user has 100% control if they'd rather specify them. Just like your current approach, the user has the ability to specify NCFLIBS. They can edit the Makefile or supply it via command-line. In the proposed approach, the only difference is that a "sane environment" would build in one line make configure && make build without edits.

There is good reason for the Makefile.templates

If you don't want a Makefile in the top-level, the if blocks above could be in Makefile.template.

To summarize, I think this edit will lead to more people doing more builds of I/O API. I have to believe that will be good for your software.

[cjcoats]

I'll add this to I/O API 4.0 (currently in progress: all free-form source, new compiler-sets, configured to use netCDF-4/HDF interfaces and libraries by default).