Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2322-solrCloud integration #2985

Closed
wants to merge 176 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
176 commits
Select commit Hold shift + click to select a range
b871a52
End of day: Semi-Stable
bionary Feb 4, 2016
3977e76
Start of day: Semi-Stable
bionary Feb 10, 2016
19c8289
END of day: Semi-Stable
bionary Feb 10, 2016
4a39415
End of Day commit 2/10/16. First steps to add solrCloud configuration…
bionary Feb 10, 2016
859f859
More End of day commits after adding forgotten orphens.
bionary Feb 10, 2016
c4c078b
Merge branch '2322-solrCloud' of github.com:bionary/dataverse into 23…
bionary Feb 11, 2016
8be553c
Source'able script to coordinate cmdline configurations
bionary Feb 11, 2016
f03c011
Modified to call an independent setup-solr.sh script
bionary Feb 11, 2016
d72fba1
Added getopts arg parsing and modifiable scripts/installer/install call
bionary Feb 11, 2016
ad23836
Vagrant setup to test solrCloud integration
bionary Feb 11, 2016
dcfb7fa
Modified to use multiple argument parsing
bionary Feb 11, 2016
34f3872
Added solr configuration options to pass through to install perlscript
bionary Feb 11, 2016
e93d1fe
Added solr collection name option to solrCloud configuration
bionary Feb 11, 2016
8ab4c12
Added solr server / solrCloud configuration options
bionary Feb 11, 2016
fc186d2
First take on adding solr initialization configurations
bionary Feb 11, 2016
52164f1
Merge remote-tracking branch 'upstream/develop' into 2322-solrCloud
bionary Feb 11, 2016
009f431
Installation and Setup script cleanup
bionary Feb 11, 2016
93fb2e6
Silence curl output in setup script
bionary Feb 12, 2016
83011c2
Renamed the vagrant test platform for testing Cloudera Hadoop CDH int…
bionary Feb 12, 2016
f2d5994
added description of what this directory is for / contains
bionary Feb 12, 2016
64cba31
minor update to coordinate static IP addresses among multiple test pl…
bionary Feb 12, 2016
c391e5f
Shifting primary vagrant VM to dataverse VM
bionary Feb 12, 2016
bbd930e
First draft of solr installation script
bionary Feb 12, 2016
74882f7
vagrant environment for dataverse to remote solr testing
bionary Feb 12, 2016
106e9cd
Merge origin/2322-solrCloud into 2322-solrCloud
bionary Feb 12, 2016
207a4e9
minor syntax fix for zookeeper getopts flag
bionary Feb 12, 2016
22388e0
Refactoring name/location of utility script for bash parsing command …
bionary Feb 13, 2016
dcb3983
Added option to adjust output verbosity of shell scripts
bionary Feb 13, 2016
d302d9c
Added solr archive files to ignore
bionary Feb 13, 2016
c4330ff
Created a source-able script to control the verbosity of shell script…
bionary Feb 13, 2016
c8a3321
Created a script to install solr and provision it as a system service
bionary Feb 13, 2016
ab22ee1
More changes to enable automated solr install and setup
bionary Feb 16, 2016
3ca8de9
added log directory setting/option
bionary Feb 16, 2016
5b626fd
added log directory setting/option
bionary Feb 16, 2016
22d976b
Fixed missing solr log directory option
bionary Feb 16, 2016
eba3372
Adjusted solr server static ip to coordinate multiple test platform i…
bionary Feb 16, 2016
c7aaa03
Added vagrant setup for remote solr to dataverse testing using the le…
bionary Feb 16, 2016
49cb421
Merge origin/2322-solrCloud into 2322-solrCloud
bionary Feb 16, 2016
6fb0c6b
Added SolrHostColonPort dataverse server configuration
bionary Feb 16, 2016
88654aa
Set default verbosity to level 3 (most verbose)
bionary Feb 16, 2016
e9d292d
Stabilized the legacy solrRemote to Dataverse vagrant setup
bionary Feb 16, 2016
c1ceae0
changed incorrect setup-all.sh parameter for solr host
bionary Feb 16, 2016
51f3540
Added inline provision to shutdown the VM firewall
bionary Feb 16, 2016
e07a567
Updated verbosity level code/settings to control output levels from i…
bionary Feb 16, 2016
f2604fb
Renamed vagrant environment to reflect solr installed and configured …
bionary Feb 17, 2016
757a4af
Updated vagrant environments to live on separate private networks
bionary Feb 17, 2016
6ce1d67
renamed solr installation script
bionary Feb 17, 2016
2437831
Changes to maintain verbose output in original/root vagrant environment
bionary Feb 17, 2016
5e67e0e
Added missing output_verbosity option "v"
bionary Feb 17, 2016
e26237f
Fixed brace error in echo message
bionary Feb 17, 2016
748a7e6
renamed variable representing dataverse api entrypoint
bionary Feb 17, 2016
e1d4881
Added provision tags to help isolate debugging
bionary Feb 17, 2016
a8c369e
Added quotes to stringify verbosity parameter to setup-all.sh script
bionary Feb 17, 2016
9d7c2b3
fixed getOpt string requirement for verbosity parameter
bionary Feb 17, 2016
290000d
Cleaned up source'd reference to the util-set-verbosity script
bionary Feb 17, 2016
c213620
added verbosity setting for call to install-dataverse.sh script
bionary Feb 17, 2016
06c1af5
added verbosity setting for call to setup.sh script
bionary Feb 17, 2016
842e0ab
fixed definition of GLASSFISH_USER_HOME for non-'glassfish' alternati…
bionary Feb 17, 2016
eb1df19
updated to reflect the division in 4.x to 5.x solr generations
bionary Feb 17, 2016
dfb1829
updated to use the solr 5.x install as a service scripts
bionary Feb 17, 2016
374d2dd
Added SolrServiceSlashCollection setting to support Solr 5.x+
bionary Feb 18, 2016
f36b8d8
Added SolrServiceSlashCollection setting to URL string for SolrServer…
bionary Feb 18, 2016
0be51b7
Added to provision vagrant environment with published root dataverse …
bionary Feb 18, 2016
454b4b7
allows vagrant provisioning of custom solr collection name for dataverse
bionary Feb 18, 2016
e8d3316
support for automated solr collection configuration with solr 5.x
bionary Feb 18, 2016
e752055
support for automated solr collection configuration with solr 5.x
bionary Feb 18, 2016
012ff50
support for automated solr collection configuration with solr 5.x
bionary Feb 18, 2016
df55085
support for automated solr collection configuration with solr 5.x
bionary Feb 18, 2016
a24424d
support for automated solr collection configuration with solr 5.x
bionary Feb 18, 2016
8d59d02
support for automated solr collection configuration with solr 5.x
bionary Feb 18, 2016
ee5dadf
updated description of vagrant test environment
bionary Feb 18, 2016
ea823e5
Updated vagrant environment descriptions to include subnet and static…
bionary Feb 18, 2016
f95e751
Added solrCloud vagrant environment
bionary Feb 19, 2016
323a19a
Added znode to zookeeper ensemble to isolate solrCloud configurations
bionary Feb 19, 2016
0aab3c1
Corrected default zookeeper port
bionary Feb 19, 2016
9ae282e
Added comment to leave zookeeper ensemble uninitialized
bionary Feb 19, 2016
c53c769
Added solrCloud zookeeper configuration API endpoints
bionary Feb 19, 2016
b2f6248
changed boolean parameter ':useSolrCloudViaZookeeper' from 'true' to …
bionary Feb 19, 2016
c54087b
added quoting to protect command line arguments to installer
bionary Feb 19, 2016
120dbbf
added solrCloud zookeeper related settings and configurations
bionary Feb 19, 2016
60b2060
added CloudSolrServer implementation option for solrServer
bionary Feb 19, 2016
e6d9013
Split settings for SolrServiceName and SolrCollectionName
bionary Feb 19, 2016
a3aa207
Split settings for SolrServiceName and SolrCollectionName
bionary Feb 19, 2016
bb34bec
Added selection of default collection for solrCloud based collections
bionary Feb 19, 2016
dc4d79f
updated configuration of solr collection name to reflect API change
bionary Feb 19, 2016
b8d980d
Added shard and replication parameters to test distributed indexing
bionary Feb 19, 2016
8ab2c93
Added SOLR_HOST to solr.in.sh setup to help with multihomed VMs
bionary Feb 19, 2016
5d7cb6d
touch ups to the '/bin/solr create' call
bionary Feb 19, 2016
969d19d
setting verbosity level for createDataverseCollection provisioner
bionary Feb 19, 2016
09ee32c
cleanup default dataverse host
bionary Feb 19, 2016
57ed6f0
cleanup default dataverse host
bionary Feb 20, 2016
0af5224
bionary Feb 20, 2016
9860ffd
bionary Feb 20, 2016
7d5982c
bionary Feb 20, 2016
f1e9d8c
generalized setup script to find data directory using either '/datave…
bionary Feb 20, 2016
ec8b8f3
added provisioner to load Pete/Uma mock dataverse tests
bionary Feb 20, 2016
078cefa
removed redundant provisioner script in favor of common script in vag…
bionary Feb 20, 2016
21aa878
removed redundant provisioner script in favor of common script in vag…
bionary Feb 20, 2016
5716968
removed redundant provisioner script in favor of common script in vag…
bionary Feb 20, 2016
977256f
removed redundant provisioner script in favor of common script in vag…
bionary Feb 20, 2016
f56b8ee
removed redundant provisioner script in favor of common script in vag…
bionary Feb 20, 2016
ee64b84
moved redundant provisioner scripts to common vagrant directory
bionary Feb 20, 2016
6b75856
moved redundant provisioner scripts to common vagrant directory
bionary Feb 20, 2016
969a4fb
support for solr TLS/SSL connections
bionary Feb 21, 2016
aa54bd1
bug fix on /bin/solr call
bionary Feb 21, 2016
be32d4f
added solr runas user
bionary Feb 21, 2016
1306805
refactored to support solrj 5.4.1 update
bionary Feb 21, 2016
8fd35fa
refactored to support solrj 5.4.1 update
bionary Feb 21, 2016
e3f5881
bionary Feb 21, 2016
3467928
bionary Feb 21, 2016
a0e91ad
makes more sense to use java from JAVA_HOME over PATH
bionary Feb 21, 2016
5e2a2d1
First attempts at setup/support of TLS/SSL connections to solr
bionary Feb 22, 2016
8137f0b
First attempts at setup/support of TLS/SSL connections to solr
bionary Feb 22, 2016
6ac9044
First attempts at setup/support of TLS/SSL connections to solr
bionary Feb 22, 2016
cb266c0
First attempts at setup/support of TLS/SSL connections to solr
bionary Feb 22, 2016
1bbb563
Cleaned up vagrant environments by restructuring common scripts
bionary Feb 22, 2016
70c162c
Refactored to update solrj libraries to 5.4.1 release
bionary Feb 22, 2016
4d75e8f
bionary Feb 22, 2016
16506c0
bionary Feb 22, 2016
20b1b3d
bionary Feb 22, 2016
9e2d317
bionary Feb 22, 2016
0e3d3bf
bionary Feb 22, 2016
e25c556
bionary Feb 22, 2016
6fbdcf9
Working option to connect to Solr via https
bionary Feb 23, 2016
09362e3
Added installer option and dataverse setting to connect to Solr via h…
bionary Feb 23, 2016
3bccc48
Added installer option and dataverse setting to connect to Solr via h…
bionary Feb 23, 2016
9b9ffbc
bionary Feb 23, 2016
96deff1
Added option to avoid failed certificate check when TLS/SSL is enabled
bionary Feb 23, 2016
6ee4f00
fixed lazy loaded variable assignment error in Vagrant provisioning loop
bionary Feb 23, 2016
a43e420
bionary Feb 23, 2016
f324e1c
final bug squashes to dataverse + solrCloud (https) vagrant setups
bionary Feb 23, 2016
070c7c9
first steps to configure a kerberos+solrCloud+dataverse test environment
bionary Feb 24, 2016
4141b7f
env variable adjustment of output verbosity added
bionary Feb 24, 2016
5c35a54
added kerberos configuration for zookeeper
bionary Feb 24, 2016
6019c53
added kerberos configuration for zookeeper
bionary Feb 24, 2016
72d4c72
added kerberos configuration for zookeeper
bionary Feb 24, 2016
058b5b3
added kerberos configuration for solr
bionary Feb 24, 2016
c8c7b4c
added kerberos configuration for solr
bionary Feb 24, 2016
58600cf
added option to simply generate the keytab
bionary Feb 24, 2016
070cd65
fixed missing newlines in solr.in.sh
bionary Feb 24, 2016
f53b399
installation of krb5 on client workstations
bionary Feb 24, 2016
6df43cb
bionary Feb 24, 2016
ab16df5
bionary Feb 24, 2016
f547b48
configuration of solrj jaas(kerberos) for dataverse
bionary Feb 24, 2016
fc4a70e
configuration of solrj jaas(kerberos) for dataverse
bionary Feb 24, 2016
5ec7f3b
Enabling kerberos authentication to solr
bionary Feb 24, 2016
f6c10bb
bionary Feb 25, 2016
606b5c3
bionary Feb 26, 2016
539849c
bionary Feb 26, 2016
0970eb3
Enabling kerberos authentication to solr
bionary Feb 29, 2016
4ca28de
bionary Feb 29, 2016
f79d9f0
bionary Mar 1, 2016
e0fda16
bionary Mar 1, 2016
d500733
bionary Mar 1, 2016
8099265
bionary Mar 1, 2016
eb44cc4
added test provisioning
bionary Mar 1, 2016
41366b3
added test provisioning
bionary Mar 1, 2016
2df4269
restructured solr kerberos toggle to use admin endpoint instead of se…
bionary Mar 1, 2016
47273f5
bionary Mar 1, 2016
49677fb
cleanup of vagrant test environments
bionary Mar 1, 2016
0bb14da
Merge pull request #2 from IQSS/develop
bionary Mar 1, 2016
6fb8976
updated vagrant environment descriptions
bionary Mar 1, 2016
dfb0148
Merge origin/2322-solrCloud into 2322-solrCloud
bionary Mar 1, 2016
1ed8680
refactored redundant setup script
bionary Mar 1, 2016
666efee
refactored redundant znode setup script
bionary Mar 1, 2016
8333d95
refactored redundant zookeeper install script
bionary Mar 1, 2016
9ca95ea
solr server needs access to a truststore prior to it being fully prov…
bionary Mar 1, 2016
941000f
work around for failed first root/publish when using kerberized solr
bionary Mar 1, 2016
6e851ea
work around for failed first root/subDVcreate when using kerberized solr
bionary Mar 1, 2016
4530382
Added TestSharding env variable check to study effect of solr index s…
bionary Mar 2, 2016
fa0813d
added default field to /select requestHandler
bionary Mar 3, 2016
4d90a1c
Merge pull request #3 from IQSS/develop
bionary Mar 15, 2016
a50b153
minor fix to provision script to check for a result key
bionary Mar 16, 2016
d8ae42d
Merge branch 'develop' as of commit 90cb6da(4/1/2016) of https://gith…
bionary Apr 6, 2016
5905776
Merge branch 'IQSS-develop' as of 4/1/2016 commit IQSS/develop 90cb6d…
bionary Apr 6, 2016
ea2953a
Merge pull request #5 from IQSS/develop
bionary May 11, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions Vagrantfile
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,9 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
v.memory = 2048
v.cpus = 1
end
config.vm.provision "shell", path: "scripts/vagrant/setup.sh"
config.vm.provision "shell", path: "scripts/vagrant/setup-solr.sh"
config.vm.provision "shell", path: "scripts/vagrant/install-dataverse.sh", args: mailserver
config.vm.provision "setupPrereqs", type: "shell", path: "scripts/vagrant/setup.sh", args: "-v 3"
config.vm.provision "setupSolr", type: "shell", path: "scripts/vagrant/setup-solr.sh"
config.vm.provision "installDataverse", type: "shell", path: "scripts/vagrant/install-dataverse.sh", args: "-m #{mailserver} -v 3"
# FIXME: get tests working and re-enable them!
#config.vm.provision "shell", path: "scripts/vagrant/test.sh"

Expand Down
1 change: 1 addition & 0 deletions conf/solr/5.x/_rest_managed.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"initArgs":{},"managedList":[]}
67 changes: 67 additions & 0 deletions conf/solr/5.x/currency.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
<?xml version="1.0" ?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

<!-- Example exchange rates file for CurrencyField type named "currency" in example schema -->

<currencyConfig version="1.0">
<rates>
<!-- Updated from http://www.exchangerate.com/ at 2011-09-27 -->
<rate from="USD" to="ARS" rate="4.333871" comment="ARGENTINA Peso" />
<rate from="USD" to="AUD" rate="1.025768" comment="AUSTRALIA Dollar" />
<rate from="USD" to="EUR" rate="0.743676" comment="European Euro" />
<rate from="USD" to="BRL" rate="1.881093" comment="BRAZIL Real" />
<rate from="USD" to="CAD" rate="1.030815" comment="CANADA Dollar" />
<rate from="USD" to="CLP" rate="519.0996" comment="CHILE Peso" />
<rate from="USD" to="CNY" rate="6.387310" comment="CHINA Yuan" />
<rate from="USD" to="CZK" rate="18.47134" comment="CZECH REP. Koruna" />
<rate from="USD" to="DKK" rate="5.515436" comment="DENMARK Krone" />
<rate from="USD" to="HKD" rate="7.801922" comment="HONG KONG Dollar" />
<rate from="USD" to="HUF" rate="215.6169" comment="HUNGARY Forint" />
<rate from="USD" to="ISK" rate="118.1280" comment="ICELAND Krona" />
<rate from="USD" to="INR" rate="49.49088" comment="INDIA Rupee" />
<rate from="USD" to="XDR" rate="0.641358" comment="INTNL MON. FUND SDR" />
<rate from="USD" to="ILS" rate="3.709739" comment="ISRAEL Sheqel" />
<rate from="USD" to="JPY" rate="76.32419" comment="JAPAN Yen" />
<rate from="USD" to="KRW" rate="1169.173" comment="KOREA (SOUTH) Won" />
<rate from="USD" to="KWD" rate="0.275142" comment="KUWAIT Dinar" />
<rate from="USD" to="MXN" rate="13.85895" comment="MEXICO Peso" />
<rate from="USD" to="NZD" rate="1.285159" comment="NEW ZEALAND Dollar" />
<rate from="USD" to="NOK" rate="5.859035" comment="NORWAY Krone" />
<rate from="USD" to="PKR" rate="87.57007" comment="PAKISTAN Rupee" />
<rate from="USD" to="PEN" rate="2.730683" comment="PERU Sol" />
<rate from="USD" to="PHP" rate="43.62039" comment="PHILIPPINES Peso" />
<rate from="USD" to="PLN" rate="3.310139" comment="POLAND Zloty" />
<rate from="USD" to="RON" rate="3.100932" comment="ROMANIA Leu" />
<rate from="USD" to="RUB" rate="32.14663" comment="RUSSIA Ruble" />
<rate from="USD" to="SAR" rate="3.750465" comment="SAUDI ARABIA Riyal" />
<rate from="USD" to="SGD" rate="1.299352" comment="SINGAPORE Dollar" />
<rate from="USD" to="ZAR" rate="8.329761" comment="SOUTH AFRICA Rand" />
<rate from="USD" to="SEK" rate="6.883442" comment="SWEDEN Krona" />
<rate from="USD" to="CHF" rate="0.906035" comment="SWITZERLAND Franc" />
<rate from="USD" to="TWD" rate="30.40283" comment="TAIWAN Dollar" />
<rate from="USD" to="THB" rate="30.89487" comment="THAILAND Baht" />
<rate from="USD" to="AED" rate="3.672955" comment="U.A.E. Dirham" />
<rate from="USD" to="UAH" rate="7.988582" comment="UKRAINE Hryvnia" />
<rate from="USD" to="GBP" rate="0.647910" comment="UNITED KINGDOM Pound" />

<!-- Cross-rates for some common currencies -->
<rate from="EUR" to="GBP" rate="0.869914" />
<rate from="EUR" to="NOK" rate="7.800095" />
<rate from="GBP" to="NOK" rate="8.966508" />
</rates>
</currencyConfig>
54 changes: 54 additions & 0 deletions conf/solr/5.x/lang/stopwords_en.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# a couple of test stopwords to test that the words are really being
# configured from this file:
stopworda
stopwordb

# Standard english stop words taken from Lucene's StopAnalyzer
a
an
and
are
as
at
be
but
by
for
if
in
into
is
it
no
not
of
on
or
such
that
the
their
then
there
these
they
this
to
was
will
with
21 changes: 21 additions & 0 deletions conf/solr/5.x/protwords.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#-----------------------------------------------------------------------
# Use a protected word file to protect against the stemmer reducing two
# unrelated words to the same base word.

# Some non-words that normally won't be encountered,
# just to test that they won't be stemmed.
dontstems
zwhacky

Loading