<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/">
    <channel>
        <title>ClickHouse Knowledge Base Feed</title>
        <link>https://clickhouse.com/docs/knowledgebase</link>
        <description>Feed of articles posted to the ClickHouse Knowledge Base</description>
        <lastBuildDate>Thu, 11 Dec 2025 16:17:59 GMT</lastBuildDate>
        <docs>https://validator.w3.org/feed/docs/rss2.html</docs>
        <generator>https://github.com/jpmonette/feed</generator>
        <language>en</language>
        <copyright>Copyright &amp;copy; 2016&amp;ndash;2025 ClickHouse, Inc. ClickHouse Docs provided under the Creative Commons CC BY-NC-SA 4.0 license. ClickHouse&amp;reg; is a registered trademark of ClickHouse, Inc.</copyright>
        <item>
            <title><![CDATA[Terraform example on how to use Cloud API]]></title>
            <link>https://clickhouse.com/docs/knowledgebase/terraform_example</link>
            <guid>https://clickhouse.com/docs/knowledgebase/terraform_example</guid>
            <pubDate>Thu, 11 Dec 2025 16:17:59 GMT</pubDate>
            <description><![CDATA[This covers an example of how you can use terraform to create/delete clusters using the API]]></description>
            <content:encoded><![CDATA[This covers an example of how you can use terraform to create/delete clusters using the API<!-- -->
<!-- -->
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="question">Question<a href="https://clickhouse.com/docs/knowledgebase/terraform_example#question" class="hash-link" aria-label="Direct link to Question" title="Direct link to Question">​</a></h2>
<p>How can I use API to manage clusters on ClickHouse Cloud?</p>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="answer">Answer<a href="https://clickhouse.com/docs/knowledgebase/terraform_example#answer" class="hash-link" aria-label="Direct link to Answer" title="Direct link to Answer">​</a></h2>
<p>We will use Terraform to configure our infra and <a href="https://registry.terraform.io/providers/ClickHouse/clickhouse/latest/docs" target="_blank" rel="noopener noreferrer">ClickHouse Provider</a></p>
<p>Steps:</p>
<p>1). Create an API Key on Cloud.
Follow the docs here - <a href="https://clickhouse.com/docs/cloud/manage/openapi" target="_blank" rel="noopener noreferrer">https://clickhouse.com/docs/cloud/manage/openapi</a></p>
<p>Save the creds locally.</p>
<p>2). Install Terraform using - <a href="https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli" target="_blank" rel="noopener noreferrer">https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli</a></p>
<p>You can use Homebrew package manager if you're on Mac.</p>
<p>3). Create a directory anywhere you like:</p>
<div class="wrapper_EBtA" style="height:56.550000000000004px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">mkdir test
➜  test pwd
/Users/jaijhala/Desktop/terraform/test
</code></pre></div><div class="activity_PoTP"></div></div>
<p>4). Create 2 files: <code>main.tf</code> and <code>secret.tfvars</code></p>
<p>Copy the following:</p>
<p><code>main.tf</code> file would be:</p>
<div class="wrapper_EBtA" style="height:980.2px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">terraform {
 required_providers {
   clickhouse = {
     source = "ClickHouse/clickhouse"
     version = "0.0.2"
   }
 }
}

variable "organization_id" {
  type = string
}

variable "token_key" {
  type = string
}

variable "token_secret" {
  type = string
}

provider clickhouse {
  environment 	= "production"
  organization_id = var.organization_id
  token_key   	= var.token_key
  token_secret	= var.token_secret
}


variable "service_password" {
  type = string
  sensitive   = true
}

resource "clickhouse_service" "service123" {
  name       	= "jai-terraform"
  cloud_provider = "aws"
  region     	= "us-east-2"
  tier       	= "development"
  idle_scaling   = true
  password  = var.service_password
  ip_access = [
	{
    	source  	= "0.0.0.0/0"
    	description = "Anywhere"
	}
  ]
}

output "CLICKHOUSE_HOST" {
  value = clickhouse_service.service123.endpoints.0.host
}
</code></pre></div><div class="activity_PoTP"></div></div>
<p>You can replace your own parameters like service name, region etc.. in the resources section above.</p>
<p><code>secret.tfvars</code> is where you'll put all the API Key related info that you downloaded earlier. The idea behind this file is that all your secret credentials will be hidden from the main config file.</p>
<p>It would be something like (replace these parameters):</p>
<div class="wrapper_EBtA" style="height:75.4px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">organization_id = "e957a5f7-4qe3-4b05-ad5a-d02b2dcd0593"
token_key = "QWhhkMeytqQruTeKg"
token_secret = "4b1dNmjWdLUno9lXxmKvSUcPP62jvn7irkuZPbY"
service_password = "password123!"
</code></pre></div><div class="activity_PoTP"></div></div>
<p>5).  Run <code>terraform init</code> from this directory</p>
<p>Expected output:</p>
<div class="wrapper_EBtA" style="height:471.25000000000006px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">Initializing the backend...

Initializing provider plugins...
- Finding clickhouse/clickhouse versions matching "0.0.2"...
- Installing clickhouse/clickhouse v0.0.2...
- Installed clickhouse/clickhouse v0.0.2 (self-signed, key ID D7089EE5C6A92ED1)

Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
</code></pre></div><div class="activity_PoTP"></div></div>
<p>6). Run <code>terraform apply -var-file=secret.tfvars</code> command.</p>
<p>Something like:</p>
<div class="wrapper_EBtA" style="height:697.45px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">➜  test terraform apply -var-file=secret.tfvars

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with
the following symbols:
  + create

Terraform will perform the following actions:

  # clickhouse_service.service123 will be created
  + resource "clickhouse_service" "service123" {
      + cloud_provider = "aws"
      + endpoints      = (known after apply)
      + id             = (known after apply)
      + idle_scaling   = true
      + ip_access      = [
          + {
              + description = "Anywhere"
              + source      = "0.0.0.0/0"
            },
        ]
      + last_updated   = (known after apply)
      + name           = "jai-terraform"
      + password       = (sensitive value)
      + region         = "us-east-2"
      + tier           = "development"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + CLICKHOUSE_HOST = (known after apply)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes
</code></pre></div><div class="activity_PoTP"></div></div>
<p>Type <code>yes</code> and hit enter</p>
<p>Side note: Notice it says <code>password       = (sensitive value)</code> above.
This is because we set <code>sensitive   = true</code> for the password in the main.tf file.</p>
<p>7). It will take a couple of mins to create the service but eventually it should come up like:</p>
<div class="wrapper_EBtA" style="height:395.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">  Enter a value: yes

clickhouse_service.service123: Creating...
clickhouse_service.service123: Still creating... [10s elapsed]
clickhouse_service.service123: Still creating... [20s elapsed]
clickhouse_service.service123: Still creating... [30s elapsed]
clickhouse_service.service123: Still creating... [40s elapsed]
clickhouse_service.service123: Still creating... [50s elapsed]
clickhouse_service.service123: Still creating... [1m0s elapsed]
clickhouse_service.service123: Still creating... [1m10s elapsed]
clickhouse_service.service123: Still creating... [1m20s elapsed]
clickhouse_service.service123: Still creating... [1m30s elapsed]
clickhouse_service.service123: Still creating... [1m40s elapsed]
clickhouse_service.service123: Creation complete after 1m41s [id=aa8d8d63-1878-4600-8470-630715af38ed]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

CLICKHOUSE_HOST = "h3ljlaqez6.us-east-2.aws.clickhouse.cloud"
➜  test
</code></pre></div><div class="activity_PoTP"></div></div>
<p>8). Check Cloud Console, you should be able to see the service created.</p>
<p>9). To clean up/destroy the service again, run <code>terraform destroy -var-file=secret.tfvars</code></p>
<p>Something like:</p>
<div class="wrapper_EBtA" style="height:395.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with
the following symbols:
  - destroy

Terraform will perform the following actions:

  # clickhouse_service.service123 will be destroyed
  - resource "clickhouse_service" "service123" {
      - cloud_provider = "aws" -&gt; null
      - ............

Plan: 0 to add, 0 to change, 1 to destroy.

Changes to Outputs:
  - CLICKHOUSE_HOST = "h3ljlaqez6.us-east-2.aws.clickhouse.cloud" -&gt; null

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value:
</code></pre></div><div class="activity_PoTP"></div></div>
<p>Type yes and hit enter</p>
<p>10).</p>
<div class="wrapper_EBtA" style="height:113.10000000000001px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">clickhouse_service.service123: Destroying... [id=aa8d8d63-1878-4600-8470-630715af38ed]
clickhouse_service.service123: Still destroying... [id=aa8d8d63-1878-4600-8470-630715af38ed, 10s elapsed]
clickhouse_service.service123: Still destroying... [id=aa8d8d63-1878-4600-8470-630715af38ed, 20s elapsed]
clickhouse_service.service123: Destruction complete after 27s

Destroy complete! Resources: 1 destroyed.
</code></pre></div><div class="activity_PoTP"></div></div>
<p>And it should be gone from the Cloud Console.</p>
<p>More details about the Cloud API can be found here - <a href="https://clickhouse.com/docs/cloud/manage/api/api-overview" target="_blank" rel="noopener noreferrer">https://clickhouse.com/docs/cloud/manage/api/api-overview</a></p><div class="container_iJTo margin-vert--md"><time datetime="2025-12-11T16:17:59.000Z">Dec 11, 2025</time> · <!-- -->5 min read</div>]]></content:encoded>
            <category>Native Clients and Interfaces</category>
        </item>
        <item>
            <title><![CDATA[How to restore a replica after storage failure]]></title>
            <link>https://clickhouse.com/docs/knowledgebase/restore-replica-after-storage-failure</link>
            <guid>https://clickhouse.com/docs/knowledgebase/restore-replica-after-storage-failure</guid>
            <pubDate>Wed, 19 Nov 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[This article explains how to recover data when using replicated tables in atomic databases in ClickHouse and disks/storage on one of the replica is lost/currupted.]]></description>
            <content:encoded><![CDATA[This article explains how to recover data when using replicated tables in atomic databases in ClickHouse and disks/storage on one of the replica is lost/currupted.<!-- -->
<!-- -->
<br>
<br>
<div class="theme-admonition theme-admonition-note alert alert--secondary admonition_WoCw"><div class="alert-icon admonitionIcon_Ibzs"><svg viewBox="0 0 14 16"><path fill-rule="evenodd" d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></div><div class="alert-content admonitionContent_vXIg"><div class="admonitionHeading_TMsN">Note</div><p>This guide assumes that the <code>&lt;path&gt;</code> parameter in your config.xml file is set to:</p><div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">&lt;path&gt;/var/lib/clickhouse/&lt;/path&gt;
</code></pre></div><div class="activity_PoTP"></div></div><p>If you have configured a different data path, replace all instances of <code>/var/lib/clickhouse</code> in the below commands with the actual value of your <code>&lt;path&gt;</code> setting.</p></div></div>
<div class="sc-xro1ly-0 QlHdI vertical-stepper "><div class="sc-xro1ly-1 bHdCRJ"><button id="step-0" class="sc-xro1ly-2 dOmWto"><div class="sc-xro1ly-3 buAqWx"></div><div class="sc-xro1ly-5 kxqSvA">Copy access configuration from the healthy replica</div></button><div class="sc-xro1ly-6 EHzUo"><h2 class="anchor anchorWithStickyNavbar_LWe7" id="copy-access-config">Copy access configuration from the healthy replica<a href="https://clickhouse.com/docs/knowledgebase/restore-replica-after-storage-failure#copy-access-config" class="hash-link" aria-label="Direct link to Copy access configuration from the healthy replica" title="Direct link to Copy access configuration from the healthy replica">​</a></h2><p>Copy the contents of the <code>access</code> folder which contains local users from the healthy replica:</p><div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">/var/lib/clickhouse/access
</code></pre></div><div class="activity_PoTP"></div></div></div></div><div class="sc-xro1ly-1 bHdCRJ"><button id="step-1" class="sc-xro1ly-2 dOmWto"><div class="sc-xro1ly-3 buAqWx"></div><div class="sc-xro1ly-5 kxqSvA">Back up the metadata folder from the healthy replica</div></button><div class="sc-xro1ly-6 EHzUo"><h2 class="anchor anchorWithStickyNavbar_LWe7" id="back-up-the-metadata-folder-from-the-healthy-replica">Back up the metadata folder from the healthy replica<a href="https://clickhouse.com/docs/knowledgebase/restore-replica-after-storage-failure#back-up-the-metadata-folder-from-the-healthy-replica" class="hash-link" aria-label="Direct link to Back up the metadata folder from the healthy replica" title="Direct link to Back up the metadata folder from the healthy replica">​</a></h2><ol class="custom-ol">
<li class="custom-li">Navigate to the ClickHouse data directory:</li>
</ol><div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">cd /var/lib/clickhouse
</code></pre></div><div class="activity_PoTP"></div></div><ol class="custom-ol" start="2">
<li class="custom-li">Create a backup of the metadata folder (including symbolic links): The metadata directory contains DDLs for databases and tables.
The database directory has symlinks to <code>/var/lib/clickhouse/store/..</code> which contains all the table DDLs.</li>
</ol><div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-bash"><code class="language-bash">{ find metadata -type f; find metadata -type l; find metadata -type l | xargs readlink -f; } | tar -cPf backup.tar --files-from=-
</code></pre></div><div class="activity_PoTP"></div></div><div class="theme-admonition theme-admonition-note alert alert--secondary admonition_WoCw"><div class="alert-icon admonitionIcon_Ibzs"><svg viewBox="0 0 14 16"><path fill-rule="evenodd" d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></div><div class="alert-content admonitionContent_vXIg"><div class="admonitionHeading_TMsN">Note</div><p>This command ensures that both the <strong>metadata files</strong>, and the symlink architecture are preserved in the backup.</p></div></div></div></div><div class="sc-xro1ly-1 bHdCRJ"><button id="step-2" class="sc-xro1ly-2 dOmWto"><div class="sc-xro1ly-3 buAqWx"></div><div class="sc-xro1ly-5 kxqSvA">Restore the metadata on the faulty replica</div></button><div class="sc-xro1ly-6 EHzUo"><h2 class="anchor anchorWithStickyNavbar_LWe7" id="restore-the-metadata-on-the-faulty-replica">Restore the metadata on the faulty replica<a href="https://clickhouse.com/docs/knowledgebase/restore-replica-after-storage-failure#restore-the-metadata-on-the-faulty-replica" class="hash-link" aria-label="Direct link to Restore the metadata on the faulty replica" title="Direct link to Restore the metadata on the faulty replica">​</a></h2><ol class="custom-ol">
<li class="custom-li">Copy the generated <code>backup.tar</code> file to the faulty replica.</li>
<li class="custom-li">Extract it to the ClickHouse data directory:</li>
</ol><div class="wrapper_EBtA" style="height:37.7px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">cd /var/lib/clickhouse/
tar -xvPf backup.tar
</code></pre></div><div class="activity_PoTP"></div></div></div></div><div class="sc-xro1ly-1 bHdCRJ"><button id="step-3" class="sc-xro1ly-2 dOmWto"><div class="sc-xro1ly-3 buAqWx"></div><div class="sc-xro1ly-5 kxqSvA">Create the force restore flag</div></button><div class="sc-xro1ly-6 EHzUo"><h2 class="anchor anchorWithStickyNavbar_LWe7" id="create-force-restore-flag">Create the force restore flag<a href="https://clickhouse.com/docs/knowledgebase/restore-replica-after-storage-failure#create-force-restore-flag" class="hash-link" aria-label="Direct link to Create the force restore flag" title="Direct link to Create the force restore flag">​</a></h2><p>To trigger automatic data synchronization from other replicas, create the following flag:</p><div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data
</code></pre></div><div class="activity_PoTP"></div></div></div></div><div class="sc-xro1ly-1 bHdCRJ"><button id="step-4" class="sc-xro1ly-2 dOmWto"><div class="sc-xro1ly-3 buAqWx"></div><div class="sc-xro1ly-5 kxqSvA">Restart the faulty replica</div></button><div class="sc-xro1ly-6 EHzUo"><h2 class="anchor anchorWithStickyNavbar_LWe7" id="restart-faulty-replica">Restart the faulty replica<a href="https://clickhouse.com/docs/knowledgebase/restore-replica-after-storage-failure#restart-faulty-replica" class="hash-link" aria-label="Direct link to Restart the faulty replica" title="Direct link to Restart the faulty replica">​</a></h2><ol class="custom-ol">
<li class="custom-li">Restart the ClickHouse server on the faulty node.</li>
<li class="custom-li">Check the server logs, you should observe parts being downloaded from the healthy replicas:</li>
</ol><div class="wrapper_EBtA" style="height:131.95000000000002px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-bash"><code class="language-bash">2025.11.02 00:00:04.047097 [ 682 ] {} &lt;Debug&gt; analytics.events_local (...) (Fetcher): Downloading files 23
2025.11.02 00:00:04.055542 [ 682 ] {} &lt;Debug&gt; analytics.events_local (...) (Fetcher): Download of part 202511_0_0_0 onto disk disk2 finished.
2025.11.02 00:00:04.101888 [ 687 ] {} &lt;Debug&gt; warehouse.customers_local (...) (Fetcher): Downloading part 2025_0_0_1 onto disk default.
2025.11.02 00:00:04.102005 [ 687 ] {} &lt;Debug&gt; warehouse.customers_local (...) (Fetcher): Downloading files 11
2025.11.02 00:00:04.102210 [ 690 ] {} &lt;Debug&gt; warehouse.customers_local (...) (Fetcher): Downloading part 2022_0_0_1 onto disk disk1.
2025.11.02 00:00:04.102247 [ 688 ] {} &lt;Debug&gt; warehouse.customers_local (...) (Fetcher): Downloading part 2021_0_0_1 onto disk disk2.
2025.11.02 00:00:04.102331 [ 690 ] {} &lt;Debug&gt; warehouse.customers_local (...) (Fetcher): Downloading files 11
</code></pre></div><div class="activity_PoTP"></div></div></div></div></div><div class="container_iJTo margin-vert--md"><time datetime="2025-11-19T00:00:00.000Z">Nov 19, 2025</time> · <!-- -->2 min read</div>]]></content:encoded>
            <category>Deployments and Scaling</category>
        </item>
        <item>
            <title><![CDATA[How to Generate a HAR file for support]]></title>
            <link>https://clickhouse.com/docs/knowledgebase/generate-har-file</link>
            <guid>https://clickhouse.com/docs/knowledgebase/generate-har-file</guid>
            <pubDate>Tue, 11 Nov 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[A HAR (HTTP Archive) file captures the network activity in your browser. It can help our support team diagnose slow page loads, failed requests, or other network issues.]]></description>
            <content:encoded><![CDATA[A HAR (HTTP Archive) file captures the network activity in your browser. It can help our support team diagnose slow page loads, failed requests, or other network issues.<!-- -->
<!-- -->
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="from-google-chrome">From Google Chrome<a href="https://clickhouse.com/docs/knowledgebase/generate-har-file#from-google-chrome" class="hash-link" aria-label="Direct link to From Google Chrome" title="Direct link to From Google Chrome">​</a></h2>
<ol class="custom-ol">
<li class="custom-li">Open Developer Tools by pressing <kbd>F12</kbd> or <kbd>Ctrl</kbd> + <kbd>Shift</kbd> + <kbd>I</kbd> (Windows/Linux) / <kbd>Cmd</kbd> + <kbd>Option</kbd> + <kbd>I</kbd> (Mac).</li>
<li class="custom-li">Click the "Network" tab.</li>
<li class="custom-li">Reload the page and reproduce the issue.</li>
<li class="custom-li">From Developer Tools, click the download button.</li>
</ol>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="from-mozilla-firefox">From Mozilla Firefox<a href="https://clickhouse.com/docs/knowledgebase/generate-har-file#from-mozilla-firefox" class="hash-link" aria-label="Direct link to From Mozilla Firefox" title="Direct link to From Mozilla Firefox">​</a></h2>
<ol class="custom-ol">
<li class="custom-li">Open Developer Tools by pressing <kbd>F12</kbd> or <kbd>Ctrl</kbd> + <kbd>Shift</kbd> + <kbd>E</kbd> (Windows/Linux) / <kbd>Cmd</kbd> + <kbd>Option</kbd> + <kbd>E</kbd> (Mac).</li>
<li class="custom-li">Click the "Network" tab.</li>
<li class="custom-li">Reload the page and reproduce the issue.</li>
<li class="custom-li">From Developer Tools, right-click in the request list and select "Save All As HAR".</li>
</ol>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="from-microsoft-edge">From Microsoft Edge<a href="https://clickhouse.com/docs/knowledgebase/generate-har-file#from-microsoft-edge" class="hash-link" aria-label="Direct link to From Microsoft Edge" title="Direct link to From Microsoft Edge">​</a></h2>
<ol class="custom-ol">
<li class="custom-li">Open Developer Tools by pressing <kbd>F12</kbd> or <kbd>Ctrl</kbd> + <kbd>Shift</kbd> + <kbd>I</kbd> (Windows/Linux) / <kbd>Cmd</kbd> + <kbd>Option</kbd> + <kbd>I</kbd> (Mac).</li>
<li class="custom-li">Click the "Network" tab.</li>
<li class="custom-li">Reload the page and reproduce the issue.</li>
<li class="custom-li">From Developer Tools, right-click in the network list and choose "Save all as HAR with content".</li>
</ol>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="from-safari">From Safari<a href="https://clickhouse.com/docs/knowledgebase/generate-har-file#from-safari" class="hash-link" aria-label="Direct link to From Safari" title="Direct link to From Safari">​</a></h2>
<ol class="custom-ol">
<li class="custom-li">Enable Developer Tools (if not already enabled):<!-- -->
<ul class="custom-ul">
<li class="custom-li">Go to Safari &gt; Settings &gt; Advanced.</li>
<li class="custom-li">Check "Show Develop menu in menu bar" at the bottom.</li>
</ul>
</li>
<li class="custom-li">Click Develop &gt; Show Web Inspector.</li>
<li class="custom-li">Click the "Network" tab.</li>
<li class="custom-li">Reload the page and reproduce the issue.</li>
<li class="custom-li">From Developer Tools, click the "Export" button.</li>
</ol>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="sending-har-file">Sending the .HAR file<a href="https://clickhouse.com/docs/knowledgebase/generate-har-file#sending-har-file" class="hash-link" aria-label="Direct link to Sending the .HAR file" title="Direct link to Sending the .HAR file">​</a></h2>
<ol class="custom-ol">
<li class="custom-li">Rename the file to something short and descriptive (e.g., login-issue.har).</li>
<li class="custom-li">Compress the file (optional but recommended).</li>
<li class="custom-li">Attach it to your support case or email it to your assigned support contact.</li>
</ol>
<div class="theme-admonition theme-admonition-note alert alert--secondary admonition_WoCw"><div class="alert-icon admonitionIcon_Ibzs"><svg viewBox="0 0 14 16"><path fill-rule="evenodd" d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></div><div class="alert-content admonitionContent_vXIg"><div class="admonitionHeading_TMsN">Note</div><p>HAR files may contain session cookies or other sensitive data.
Only share them with authorized support personnel.</p></div></div><div class="container_iJTo margin-vert--md"><time datetime="2025-11-11T00:00:00.000Z">Nov 11, 2025</time> · <!-- -->2 min read</div>]]></content:encoded>
            <category>Tools and Utilities</category>
        </item>
        <item>
            <title><![CDATA[Memory limit exceeded for query]]></title>
            <link>https://clickhouse.com/docs/knowledgebase/memory-limit-exceeded-for-query</link>
            <guid>https://clickhouse.com/docs/knowledgebase/memory-limit-exceeded-for-query</guid>
            <pubDate>Fri, 25 Jul 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Troubleshooting memory limit exceeded errors for a query]]></description>
            <content:encoded><![CDATA[Troubleshooting memory limit exceeded errors for a query<!-- -->
<!-- -->
<!-- -->
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="troubleshooting-out-of-memory-issues">Memory limit exceeded for query<a href="https://clickhouse.com/docs/knowledgebase/memory-limit-exceeded-for-query#troubleshooting-out-of-memory-issues" class="hash-link" aria-label="Direct link to Memory limit exceeded for query" title="Direct link to Memory limit exceeded for query">​</a></h2>
<p>As a new user, ClickHouse can often seem like magic - every query is super fast,
even on the largest datasets and most ambitious queries. Invariably though,
real-world usage tests even the limits of ClickHouse. Queries exceeding memory
can be the result of a number of causes. Most commonly, we see large joins or
aggregations on high cardinality fields. If performance is critical, and these
queries are required, we often recommend users simply scale up - something
ClickHouse Cloud does automatically and effortlessly to ensure your queries
remain responsive. We appreciate, however, that in self-managed scenarios,
this is sometimes not trivial, and maybe optimal performance is not even required.
Users, in this case, have a few options.</p>
<h3 class="anchor anchorWithStickyNavbar_LWe7" id="aggregations">Aggregations<a href="https://clickhouse.com/docs/knowledgebase/memory-limit-exceeded-for-query#aggregations" class="hash-link" aria-label="Direct link to Aggregations" title="Direct link to Aggregations">​</a></h3>
<p>For memory-intensive aggregations or sorting scenarios, users can use the settings
<a href="https://clickhouse.com/docs/operations/settings/settings#max_bytes_before_external_group_by"><code>max_bytes_before_external_group_by</code></a>
and <a href="https://clickhouse.com/docs/operations/settings/settings#max_bytes_ratio_before_external_sort"><code>max_bytes_before_external_sort</code></a> respectively.
The former of which is discussed extensively <a href="https://clickhouse.com/docs/sql-reference/statements/select/group-by#group-by-in-external-memory">here</a>.</p>
<p>In summary, this ensures any aggregations can “spill” out to disk if a memory
threshold is exceeded. This will invariably impact query performance but will
help ensure queries do not OOM. The latter sorting setting helps address similar
issues with memory-intensive sorts. This can be particularly important in
distributed environments where a coordinating node receives sorted responses
from child shards. In this case, the coordinating server can be asked to sort a
dataset larger than its available memory. With <a href="https://clickhouse.com/docs/operations/settings/settings#max_bytes_ratio_before_external_sort"><code>max_bytes_before_external_sort</code></a>,
sorting can be allowed to spill over to disk. This setting is also helpful for
cases where the user has an <code>ORDER BY</code> after a <code>GROUP BY</code> with a <code>LIMIT</code>,
especially in cases where the query is distributed.</p>
<h3 class="anchor anchorWithStickyNavbar_LWe7" id="joins">Joins<a href="https://clickhouse.com/docs/knowledgebase/memory-limit-exceeded-for-query#joins" class="hash-link" aria-label="Direct link to Joins" title="Direct link to Joins">​</a></h3>
<p>For joins, users can select different <code>JOIN</code> algorithms, which can assist in
lowering the required memory. By default, joins use the hash join, which offers
the most completeness with respect to features and often the best performance.
This algorithm loads the right-hand table of the <code>JOIN</code> into an in-memory hash
table, against which the left-hand table is then evaluated. To minimize memory,
users should thus place the smaller table on the right side. This approach still
has limitations in memory-bound cases, however. In these cases, <code>partial_merge</code>
join can be enabled via the <a href="https://clickhouse.com/docs/operations/settings/settings#join_algorithm"><code>join_algorithm</code></a>
setting. This derivative of the <a href="https://en.wikipedia.org/wiki/Sort-merge_join" target="_blank" rel="noopener noreferrer">sort-merge algorithm</a>,
first sorts the right table into blocks and creates a min-max index for them.
It then sorts parts of the left table by the join key and joins them over the
right table. The min-max index is used to skip unneeded right table blocks.
This is less memory-intensive at the expense of performance. Taking this concept
further, the <code>full_sorting_merge</code> algorithm allows a <code>JOIN</code> to be performed when
the right-hand side is very large and doesn't fit into memory and lookups are
impossible, e.g. a complex subquery. In this case, both the right and left side
are sorted on disk if they do not fit in memory, allowing large tables to be
joined.</p>
<div style="position:relative;margin-bottom:16px;margin-top:16px"><div style="cursor:default"><div style="background-size:cover;background-repeat:no-repeat;position:relative;background-image:url(&quot;data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAAICAYAAADA+m62AAAACXBIWXMAAAsTAAALEwEAmpwYAAABO0lEQVR4nAXB3S5CAQDA8fMQtnCo2FBX0VZdmbJGNzbyEWutTh+k4tSWutEqLmo+xqW22lwgYcsOcsGY5UZb6QF6BO/w9/sJGs0wE7pxhtWDTE0ZcK558Xp92GwzxBMy5+dnSJIXIby1RTaTJhgM4pc8fDyk+Xx75Oq6Sqfd5va2Rr1eR0jI2xQPU4TDIVJJmb9ujp/vZy5KZbrdX2q1Go3GC4JWO8ToiBpRVGG3z5LYyxKJRrFap5HlHfL5HIGAhKDXT2A2m9HpxjAaDXjca8RiEZzORfw+N+FNHxvrKwinx0XuqhVOjo+Q5V0URaHZbFKpVPhSCnRek9xfHiDEoxLFfILNkJ9MZp9er0er1eKiVOK9XuDnaY+bcg5BFPsRxQFUqj4sFhMu1yrLy0s4HHOsL1rwr06yMG/iHyM8uOUuCkPAAAAAAElFTkSuQmCC&quot;);width:600px;margin:0 auto;display:block;box-shadow:none"><svg style="width:100%;height:auto;max-width:100%;margin-bottom:-4px" width="1024" height="768"></svg><noscript><img style=width:100%;height:auto;max-width:100%;margin-bottom:-4px;position:absolute;top:0;left:0 src=/docs/assets/ideal-img/memory-limit-exceeded-for-query.79cc359.48.png srcset="/docs/assets/ideal-img/memory-limit-exceeded-for-query.79cc359.48.png 48w,/docs/assets/ideal-img/memory-limit-exceeded-for-query.1b8b965.300.png 300w,/docs/assets/ideal-img/memory-limit-exceeded-for-query.23e1b2d.600.png 600w,/docs/assets/ideal-img/memory-limit-exceeded-for-query.7304bcc.1024.png 1024w" alt="Joins algorithms" width=1024 height=768></noscript></div></div></div>
<p>Since 20.3, ClickHouse has supported an auto value for the <code>join_algorithm</code> setting.
This instructs ClickHouse to apply an adaptive join approach, where the hash-join
algorithm is preferred until memory limits are violated, at which point the
partial_merge algorithm is attempted. Finally, concerning joins, we encourage
readers to be aware of the behavior of distributed joins and how to minimize
their memory consumption. More information can be found <a href="https://clickhouse.com/docs/sql-reference/operators/in#distributed-subqueries">here</a>.</p><div class="container_iJTo margin-vert--md"><time datetime="2025-07-25T00:00:00.000Z">Jul 25, 2025</time> · <!-- -->3 min read</div>]]></content:encoded>
            <category>Errors and Exceptions</category>
        </item>
        <item>
            <title><![CDATA[What is the difference between OPTIMIZE FINAL and FINAL?]]></title>
            <link>https://clickhouse.com/docs/knowledgebase/optimize_final_vs_final</link>
            <guid>https://clickhouse.com/docs/knowledgebase/optimize_final_vs_final</guid>
            <pubDate>Sun, 20 Jul 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Discusses the differences between OPTIMIZE FINAL and FINAL, and when to use and avoid them.]]></description>
            <content:encoded><![CDATA[Discusses the differences between OPTIMIZE FINAL and FINAL, and when to use and avoid them.<!-- -->
<!-- -->
<p><code>OPTIMIZE FINAL</code> is a DDL command that physically and permanently reorganizes
and optimizes data on disk. It physically merges data parts in <code>MergeTree</code> tables,
performing data deduplication in the process by removing duplicate rows from storage.</p>
<p><code>FINAL</code> is a <strong>query-time</strong> modifier that provides deduplicated results without
changing the structure of the stored data. It works by performing merge logic at
read-time. It is temporary, only affecting the current query result.</p>
<p>Users are often advised to avoid using <code>OPTIMIZE FINAL</code>, as it has a significant
performance overhead, however they should not confuse the two. It is often necessary
to use <code>FINAL</code> to get back results without duplicates, especially when using table
engines like <code>ReplacingMergeTree</code> which may contain duplicate rows which have not
been replaced during the eventual, background merge process.</p>
<p>The table below summarises the key differences:</p>
<table><thead><tr><th>Aspect</th><th><code>OPTIMIZE FINAL</code></th><th><code>FINAL</code></th></tr></thead><tbody><tr><td>Type</td><td>DDL Command</td><td>Query Modifier</td></tr><tr><td>Effect</td><td>Permanent storage optimization</td><td>Temporary query-time deduplication</td></tr><tr><td>Performance</td><td>Impact	High cost once, then faster queries</td><td>Lower individual cost, but repeated for each query</td></tr><tr><td>Data Modification</td><td>Yes - physically changes storage</td><td>No - read-only operation</td></tr><tr><td>Use Case</td><td>Periodic maintenance/optimization</td><td>Real-time deduplicated queries</td></tr></tbody></table>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="when-to-use-each">When to use each<a href="https://clickhouse.com/docs/knowledgebase/optimize_final_vs_final#when-to-use-each" class="hash-link" aria-label="Direct link to When to use each" title="Direct link to When to use each">​</a></h2>
<p>Use <code>OPTIMIZE FINAL</code> when:</p>
<ul class="custom-ul">
<li class="custom-li">You want to permanently improve query performance</li>
<li class="custom-li">You can afford the one-time optimization cost</li>
<li class="custom-li">You're doing periodic table maintenance</li>
<li class="custom-li">You want to physically clean up duplicate data</li>
</ul>
<p>Use <code>FINAL</code> when:</p>
<ul class="custom-ul">
<li class="custom-li">You need deduplicated results immediately</li>
<li class="custom-li">You can't wait for or don't want permanent optimization</li>
<li class="custom-li">You only occasionally need deduplicated data</li>
<li class="custom-li">You're working with frequently changing data</li>
</ul>
<p>Both are valuable tools, but they serve different purposes in ClickHouse's deduplication strategy.</p><div class="container_iJTo margin-vert--md"><time datetime="2025-07-20T00:00:00.000Z">Jul 20, 2025</time> · <!-- -->2 min read</div>]]></content:encoded>
            <category>Core Data Concepts</category>
        </item>
        <item>
            <title><![CDATA[Create a custom DNS alias by setting up a reverse proxy]]></title>
            <link>https://clickhouse.com/docs/knowledgebase/custom-dns-alias-for-instance</link>
            <guid>https://clickhouse.com/docs/knowledgebase/custom-dns-alias-for-instance</guid>
            <pubDate>Fri, 16 May 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to set up a custom DNS alias for your instance using a reverse proxy]]></description>
            <content:encoded><![CDATA[Learn how to set up a custom DNS alias for your instance using a reverse proxy<!-- -->
<!-- -->
<br>
<br>
<blockquote>
<p>In this knowledgebase article, we will walk you through how you can set up a
custom DNS alias for your ClickHouse Cloud instance through the use of a reverse
proxy such as Nginx for ClickHouse native client.</p>
</blockquote>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="create-certificate">Create a self-signed certificate<a href="https://clickhouse.com/docs/knowledgebase/custom-dns-alias-for-instance#create-certificate" class="hash-link" aria-label="Direct link to Create a self-signed certificate" title="Direct link to Create a self-signed certificate">​</a></h2>
<div class="theme-admonition theme-admonition-note alert alert--secondary admonition_WoCw"><div class="alert-icon admonitionIcon_Ibzs"><svg viewBox="0 0 14 16"><path fill-rule="evenodd" d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></div><div class="alert-content admonitionContent_vXIg"><div class="admonitionHeading_TMsN">Note</div><p>This step is not needed if you are using signed certificates.</p></div></div>
<p>Create a self-signed certificate with the domain name of your choice.
In this example we will use a domain name <code>xyz-customdomain.com</code> and
create a certificate called <code>MyCertificate.crt</code>. Refer to <a href="https://clickhouse.com/docs/guides/sre/configuring-ssl#2-create-ssl-certificates">"Create SSL certificates"</a>
for further details.</p>
<p>Add the certificate to <code>/etc/clickhouse-client/config.xml</code>:</p>
<div class="wrapper_EBtA" style="height:282.75px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-yaml"><code class="language-yaml">&lt;clickhouse&gt;
    &lt;openSSL&gt;
        &lt;client&gt;
            &lt;loadDefaultCAFile&gt;false&lt;/loadDefaultCAFile&gt;
            # highlight-next-line
            &lt;caConfig&gt;/etc/ssl/certs/MyCertificate.crt&lt;/caConfig&gt;
            &lt;cacheSessions&gt;true&lt;/cacheSessions&gt;
            &lt;disableProtocols&gt;sslv2,sslv3&lt;/disableProtocols&gt;
            &lt;preferServerCiphers&gt;true&lt;/preferServerCiphers&gt;
            &lt;invalidCertificateHandler&gt;
                &lt;name&gt;RejectCertificateHandler&lt;/name&gt;
            &lt;/invalidCertificateHandler&gt;
        &lt;/client&gt;
    &lt;/openSSL&gt;
&lt;/clickhouse&gt;
</code></pre></div><div class="activity_PoTP"></div></div>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="update-nginx-config">Update Nginx configuration<a href="https://clickhouse.com/docs/knowledgebase/custom-dns-alias-for-instance#update-nginx-config" class="hash-link" aria-label="Direct link to Update Nginx configuration" title="Direct link to Update Nginx configuration">​</a></h2>
<p>Add the following in your <code>nginx.conf</code> file:</p>
<div class="wrapper_EBtA" style="height:37.7px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">proxy_ssl_name xyz.us-west-2.aws.clickhouse.cloud;
proxy_ssl_server_name on;
</code></pre></div><div class="activity_PoTP"></div></div>
<div class="wrapper_EBtA" style="height:490.1px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">stream {
    upstream stream_backend {
         server xyz.us-west-2.aws.clickhouse.cloud:9440;
    }

    server {
        listen                9440 ssl;
        proxy_pass            stream_backend;

        ssl_certificate       /etc/ssl/certs/MyCertificate.crt;
        ssl_certificate_key   /etc/ssl/certs/MyKey.key;
        ssl_protocols         SSLv3 TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers           HIGH:!aNULL:!MD5;
        ssl_session_cache     shared:SSL:20m;
        ssl_session_timeout   4h;
        ssl_handshake_timeout 30s;
	proxy_ssl on;
	proxy_ssl_trusted_certificate /etc/ssl/certs/isrgrootx1.pem;
	proxy_ssl_session_reuse on;
        proxy_ssl_verify on;
    #highlight-next-line
	proxy_ssl_name xyz.us-west-2.aws.clickhouse.cloud;
    #highlight-next-line
	proxy_ssl_server_name on;
    }
}
</code></pre></div><div class="activity_PoTP"></div></div>
<p>Where <code>isrgrootx1.pem</code> is the root certificate for ClickHouse Cloud which you
can download <a href="https://letsencrypt.org/certs/isrgrootx1.pem" target="_blank" rel="noopener noreferrer">here</a>.</p>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="update-hosts-file">Update hosts file<a href="https://clickhouse.com/docs/knowledgebase/custom-dns-alias-for-instance#update-hosts-file" class="hash-link" aria-label="Direct link to Update hosts file" title="Direct link to Update hosts file">​</a></h2>
<div class="theme-admonition theme-admonition-note alert alert--secondary admonition_WoCw"><div class="alert-icon admonitionIcon_Ibzs"><svg viewBox="0 0 14 16"><path fill-rule="evenodd" d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></div><div class="alert-content admonitionContent_vXIg"><div class="admonitionHeading_TMsN">Note</div><p>The following step is not needed if you are using your own domain controllers</p></div></div>
<p>Add the following to your <code>/etc/hosts</code> file on the Nginx server:</p>
<div class="wrapper_EBtA" style="height:56.550000000000004px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost6 localhost6.localdomain6
10.X.Y.Z  xyz-customdomain.com
</code></pre></div><div class="activity_PoTP"></div></div>
<p>Where <code>10.X.Y.Z</code> is the IP address of your specific Nginx box.</p>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="connect-to-cloud-using-alias">Connect to Cloud using alias<a href="https://clickhouse.com/docs/knowledgebase/custom-dns-alias-for-instance#connect-to-cloud-using-alias" class="hash-link" aria-label="Direct link to Connect to Cloud using alias" title="Direct link to Connect to Cloud using alias">​</a></h2>
<p>You are now ready to connect using your custom alias:</p>
<div class="wrapper_EBtA" style="height:113.10000000000001px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-bash"><code class="language-bash">clickhouse-client --host xyz.customdomain.com --secure --password 'xxxxxxx'
ClickHouse client version 23.12.1.428 (official build).
Connecting to xyz.customdomain.com:9440 as user default.
Connected to ClickHouse server version 23.9.2.

clickhouse-cloud :)
</code></pre></div><div class="activity_PoTP"></div></div><div class="container_iJTo margin-vert--md"><time datetime="2025-05-16T00:00:00.000Z">May 16, 2025</time> · <!-- -->2 min read</div>]]></content:encoded>
            <category>Server Admin</category>
            <category>Security and Authentication</category>
        </item>
        <item>
            <title><![CDATA[AWS PrivateLink setup to expose MSK for ClickPipes]]></title>
            <link>https://clickhouse.com/docs/knowledgebase/aws-privatelink-setup-for-msk-clickpipes</link>
            <guid>https://clickhouse.com/docs/knowledgebase/aws-privatelink-setup-for-msk-clickpipes</guid>
            <pubDate>Tue, 15 Apr 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Setup steps to expose a private MSK via MSK multi-VPC connectivity to ClickPipes.]]></description>
            <content:encoded><![CDATA[Setup steps to expose a private MSK via MSK multi-VPC connectivity to ClickPipes.<!-- -->
<!-- -->
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="overview">Overview<a href="https://clickhouse.com/docs/knowledgebase/aws-privatelink-setup-for-msk-clickpipes#overview" class="hash-link" aria-label="Direct link to Overview" title="Direct link to Overview">​</a></h2>
<p>This guide will get you started with setting up a <strong>MSK multi-VPC</strong> to be used with <a href="https://clickhouse.com/docs/integrations/clickpipes/aws-privatelink#msk-multi-vpc">ClickPipes reverse private endpoint</a>.</p>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="requirements">Requirements<a href="https://clickhouse.com/docs/knowledgebase/aws-privatelink-setup-for-msk-clickpipes#requirements" class="hash-link" aria-label="Direct link to Requirements" title="Direct link to Requirements">​</a></h2>
<p>Your MSK cluster VPC must be located in one of our ClickPipes regions. See <a href="https://clickhouse.com/docs/integrations/clickpipes/aws-privatelink#aws-privatelink-regions">ClickPipes regions</a> for the list of supported regions.</p>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="enabling-multi-vpc-connectivity">Enabling multi-VPC connectivity<a href="https://clickhouse.com/docs/knowledgebase/aws-privatelink-setup-for-msk-clickpipes#enabling-multi-vpc-connectivity" class="hash-link" aria-label="Direct link to Enabling multi-VPC connectivity" title="Direct link to Enabling multi-VPC connectivity">​</a></h2>
<ol class="custom-ol">
<li class="custom-li">Navigate to the MSK cluster.<!-- -->
<ul class="custom-ul">
<li class="custom-li">Choose "Clusters" from the left navigation pane in the Amazon MSK console.</li>
<li class="custom-li">Select the specific MSK cluster you want to configure for multi-VPC connectivity.</li>
</ul>
</li>
<li class="custom-li">Enable MSK multi-VPC connectivity<!-- -->
<ul class="custom-ul">
<li class="custom-li">In the <strong>Connectivity</strong> tab, find the <strong>Multi-VPC connectivity</strong> section.</li>
<li class="custom-li">Click on <strong>Edit</strong>.</li>
<li class="custom-li">Enable the <strong>Turn-on MSK multi-VPC connectivity</strong> option.</li>
<li class="custom-li">Follow the instructions</li>
</ul>
</li>
<li class="custom-li">Add ClickPipes account principal into a cluster’s policy<!-- -->
<ul class="custom-ul">
<li class="custom-li">Navigate to the <strong>Configuration</strong> tab.</li>
<li class="custom-li">Click on <strong>Edit</strong> in the <strong>Cluster policy</strong> section.</li>
<li class="custom-li">Include <code>arn:aws:iam::072088201116:root</code> in the <strong>IAM policy</strong>. Example:<!-- -->
<div class="wrapper_EBtA" style="height:339.3px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-json"><code class="language-json">{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::072088201116:root"
                ]
            },
            "Action": [
                "kafka-cluster:Connect",
                "kafka-cluster:DescribeCluster",
                "kafka-cluster:ListClusters"
            ]
        }
    ]
}
</code></pre></div><div class="activity_PoTP"></div></div>
</li>
</ul>
</li>
</ol>
<div class="theme-admonition theme-admonition-tip alert alert--success admonition_WoCw"><div class="alert-icon admonitionIcon_Ibzs"><svg viewBox="0 0 12 16"><path fill-rule="evenodd" d="M6.5 0C3.48 0 1 2.19 1 5c0 .92.55 2.25 1 3 1.34 2.25 1.78 2.78 2 4v1h5v-1c.22-1.22.66-1.75 2-4 .45-.75 1-2.08 1-3 0-2.81-2.48-5-5.5-5zm3.64 7.48c-.25.44-.47.8-.67 1.11-.86 1.41-1.25 2.06-1.45 3.23-.02.05-.02.11-.02.17H5c0-.06 0-.13-.02-.17-.2-1.17-.59-1.83-1.45-3.23-.2-.31-.42-.67-.67-1.11C2.44 6.78 2 5.65 2 5c0-2.2 2.02-4 4.5-4 1.22 0 2.36.42 3.22 1.19C10.55 2.94 11 3.94 11 5c0 .66-.44 1.78-.86 2.48zM4 14h5c-.23 1.14-1.3 2-2.5 2s-2.27-.86-2.5-2z"></path></svg></div><div class="alert-content admonitionContent_vXIg"><div class="admonitionHeading_TMsN">Tip</div><p>Cluster IAM policy enables ClickPipes to initiate the connection from ClickPipes to your MSK cluster.
If you want to configure IAM authentication for your MSK cluster, please refer to the <a href="https://clickhouse.com/docs/integrations/clickpipes/kafka/best-practices#iam">IAM authentication documentation</a>.</p></div></div>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="creating-reverse-private-endpoint">Creating reverse private endpoint<a href="https://clickhouse.com/docs/knowledgebase/aws-privatelink-setup-for-msk-clickpipes#creating-reverse-private-endpoint" class="hash-link" aria-label="Direct link to Creating reverse private endpoint" title="Direct link to Creating reverse private endpoint">​</a></h2>
<p>Follow reverse private endpoint creation steps in the <a href="https://clickhouse.com/docs/integrations/clickpipes/aws-privatelink#creating-clickpipe">ClickPipes documentation</a>.</p><div class="container_iJTo margin-vert--md"><time datetime="2025-04-15T00:00:00.000Z">Apr 15, 2025</time> · <!-- -->2 min read</div>]]></content:encoded>
            <category>Security and Authentication</category>
            <category>Managing Cloud</category>
        </item>
        <item>
            <title><![CDATA[How do I remove the default user?]]></title>
            <link>https://clickhouse.com/docs/knowledgebase/remove-default-user</link>
            <guid>https://clickhouse.com/docs/knowledgebase/remove-default-user</guid>
            <pubDate>Thu, 16 Jan 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to remove the default user when running ClickHouse Server.]]></description>
            <content:encoded><![CDATA[Learn how to remove the default user when running ClickHouse Server.<!-- -->
<!-- -->
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="removing-the-default-user">Removing the default user<a href="https://clickhouse.com/docs/knowledgebase/remove-default-user#removing-the-default-user" class="hash-link" aria-label="Direct link to Removing the default user" title="Direct link to Removing the default user">​</a></h2>
<div class="theme-admonition theme-admonition-note alert alert--secondary admonition_WoCw"><div class="alert-icon admonitionIcon_Ibzs"><svg viewBox="0 0 14 16"><path fill-rule="evenodd" d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></div><div class="alert-content admonitionContent_vXIg"><div class="admonitionHeading_TMsN">Note</div><p>This guide is not applicable to ClickHouse Cloud.</p></div></div>
<p>In this guide, we're going to learn how to remove the <code>default</code> user from ClickHouse Server.</p>
<p>We can do this by creating a YAML file (let's call it <code>remove_default_user.yaml</code>) that has the following content</p>
<div class="wrapper_EBtA" style="height:56.550000000000004px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-yaml"><code class="language-yaml">users:
  default:
    "@remove": remove
</code></pre></div><div class="activity_PoTP"></div></div>
<p>The location of this file depends on how we have ClickHouse installed.</p>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="running-the-executable-directly">Running the executable directly<a href="https://clickhouse.com/docs/knowledgebase/remove-default-user#running-the-executable-directly" class="hash-link" aria-label="Direct link to Running the executable directly" title="Direct link to Running the executable directly">​</a></h2>
<p>If we're running the ClickHouse directly (<code>clickhouse server</code>), we need to put the file under the <code>config.d</code> directory.</p>
<p>When we run ClickHouse Server:</p>
<div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">clickhouse server
</code></pre></div><div class="activity_PoTP"></div></div>
<p>We'll see the following line in the logs:</p>
<div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">{} &lt;Debug&gt; ConfigProcessor: Merging configuration file 'config.d/remove_default_user.yaml'.
</code></pre></div><div class="activity_PoTP"></div></div>
<p>And we'll be unable to connect using <code>clickhouse client</code>:</p>
<div class="wrapper_EBtA" style="height:94.25px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">ClickHouse client version 24.11.1.2557 (official build).
Connecting to localhost:9000 as user default.
Password for user (default):
Connecting to localhost:9000 as user default.
Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: default: Authentication failed: password is incorrect, or there is no user with such name.
</code></pre></div><div class="activity_PoTP"></div></div>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="docker-or-installed">Docker or installed<a href="https://clickhouse.com/docs/knowledgebase/remove-default-user#docker-or-installed" class="hash-link" aria-label="Direct link to Docker or installed" title="Direct link to Docker or installed">​</a></h2>
<p>If we're running ClickHouse via Docker or have it installed on our machine, we need to put the file under the <code>/etc/clickhouse-server/users.d</code> directory instead.</p>
<p>So if we're running with Docker, we can mount the <code>config.d</code> directory that we created earlier to <code>/etc/clickhouse-server/users.d</code>:</p>
<div class="wrapper_EBtA" style="height:75.4px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-bash"><code class="language-bash">docker run \
  -v ./config.d:/etc/clickhouse-server/users.d \
  -p 8123:8123 -p9000:9000 \
  clickhouse/clickhouse-server:24.12
</code></pre></div><div class="activity_PoTP"></div></div>
<div class="wrapper_EBtA" style="height:56.550000000000004px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">Merging configuration file '/etc/clickhouse-server/config.d/docker_related_config.xml'.
Logging trace to /var/log/clickhouse-server/clickhouse-server.log
Logging errors to /var/log/clickhouse-server/clickhouse-server.err.log
</code></pre></div><div class="activity_PoTP"></div></div>
<p>We can then search the server logs to check that it gets picked up:</p>
<div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-bash"><code class="language-bash">docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Names}}\t{{.Command}}"
</code></pre></div><div class="activity_PoTP"></div></div>
<div class="wrapper_EBtA" style="height:37.7px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">CONTAINER ID   IMAGE                                NAMES               COMMAND
383e8ed89431   clickhouse/clickhouse-server:24.12   trusting_rosalind   "/entrypoint.sh"
</code></pre></div><div class="activity_PoTP"></div></div>
<div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">docker exec -it trusting_rosalind grep "users\.d" /var/log/clickhouse-server/clickhouse-server.log
</code></pre></div><div class="activity_PoTP"></div></div>
<p>We should see the following line:</p>
<div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">{} &lt;Debug&gt; ConfigProcessor: Merging configuration file '/etc/clickhouse-server/users.d/remove_default_user.yaml'.
</code></pre></div><div class="activity_PoTP"></div></div><div class="container_iJTo margin-vert--md"><time datetime="2025-01-16T00:00:00.000Z">Jan 16, 2025</time> · <!-- -->2 min read</div>]]></content:encoded>
            <category>Server Admin</category>
        </item>
        <item>
            <title><![CDATA[Why is ClickHouse logging so verbose by default?]]></title>
            <link>https://clickhouse.com/docs/knowledgebase/why_default_logging_verbose</link>
            <guid>https://clickhouse.com/docs/knowledgebase/why_default_logging_verbose</guid>
            <pubDate>Fri, 10 Jan 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn why the ClickHouse developers chose to set a verbose logging level by default.]]></description>
            <content:encoded><![CDATA[Learn why the ClickHouse developers chose to set a verbose logging level by default.<!-- -->
<!-- -->
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="verbose-logging">Verbose Logging<a href="https://clickhouse.com/docs/knowledgebase/why_default_logging_verbose#verbose-logging" class="hash-link" aria-label="Direct link to Verbose Logging" title="Direct link to Verbose Logging">​</a></h2>
<p>One thing that often confuses new users is that ClickHouse generates a lot of logging output, even under light load.</p>
<p>This is because the default logging level, due to historical reasons, is <code>trace</code> (instead of <code>warning</code> that would be the default in other databases).</p>
<p>The ClickHouse developers argue that <code>trace</code> provides a lot of insight in case something goes wrong.</p>
<p>On the other hand, large volumes of logging mean that the system table <code>system.text_log</code> fills up quickly and needs to be merged in the background.</p>
<p>If the database runs stably, users may re-configure the log level, which we explain how to do below.</p>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="change-the-log-level">Change the log level<a href="https://clickhouse.com/docs/knowledgebase/why_default_logging_verbose#change-the-log-level" class="hash-link" aria-label="Direct link to Change the log level" title="Direct link to Change the log level">​</a></h2>
<p>The different log levels available are documented <a href="https://clickhouse.com/docs/operations/server-configuration-parameters/settings#logger" target="_blank" rel="noopener noreferrer">here</a></p>
<p>You'll need to edit the ClickHouse server configuration file (<code>/etc/clickhouse-server/config.xml</code>) to modify the log level.</p>
<p>The default value is <code>trace</code> but you can change it to the desired level. See the comment below:</p>
<div class="wrapper_EBtA" style="height:377px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-txt"><code class="language-txt">&lt;clickhouse&gt;
    &lt;logger&gt;
        &lt;!-- Possible levels [1]:

          - none (turns off logging)
          - fatal
          - critical
          - error
          - warning
          - notice
          - information
          - debug
          - trace
          - test (not for production usage)

            [1]: https://github.com/pocoproject/poco/blob/poco-1.9.4-release/Foundation/include/Poco/Logger.h#L105-L114
        --&gt;
        &lt;level&gt;trace&lt;/level&gt;
... Rest of the configuration file
&lt;/clickhouse&gt;
</code></pre></div><div class="activity_PoTP"></div></div><div class="container_iJTo margin-vert--md"><time datetime="2025-01-10T00:00:00.000Z">Jan 10, 2025</time> · <!-- -->2 min read</div>]]></content:encoded>
            <category>Settings</category>
        </item>
        <item>
            <title><![CDATA[Adding a column to a table]]></title>
            <link>https://clickhouse.com/docs/knowledgebase/add-column</link>
            <guid>https://clickhouse.com/docs/knowledgebase/add-column</guid>
            <pubDate>Wed, 18 Dec 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[In this guide, we'll learn how to add a column to an existing table.]]></description>
            <content:encoded><![CDATA[In this guide, we'll learn how to add a column to an existing table.<!-- -->
<!-- -->
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="adding-a-column-to-a-table">Adding a Column to a Table<a href="https://clickhouse.com/docs/knowledgebase/add-column#adding-a-column-to-a-table" class="hash-link" aria-label="Direct link to Adding a Column to a Table" title="Direct link to Adding a Column to a Table">​</a></h2>
<p>We'll be using clickhouse-local:</p>
<div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-bash"><code class="language-bash">clickhouse -m --output_format_pretty_row_numbers=
</code></pre></div><div class="activity_PoTP"></div></div>
<p>Let's imagine we have the following table:</p>
<div class="wrapper_EBtA" style="height:113.10000000000001px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">CREATE TABLE events (
    date Date DEFAULT today(), 
    name String
) 
ENGINE = MergeTree
ORDER BY date;
</code></pre></div><div class="activity_PoTP"></div></div>
<p>Let's add one record:</p>
<div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">INSERT INTO events (name) VALUES ('Alexey');
</code></pre></div><div class="activity_PoTP"></div></div>
<p>And now query the <code>events</code> table:</p>
<div class="wrapper_EBtA" style="height:37.7px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">SELECT *
FROM events;
</code></pre></div><div class="activity_PoTP"></div></div>
<div class="wrapper_EBtA" style="height:56.550000000000004px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">┌───────date─┬─name───┐
│ 2024-12-18 │ Alexey │
└────────────┴────────┘
</code></pre></div><div class="activity_PoTP"></div></div>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="adding-a-new-column">Adding a new column<a href="https://clickhouse.com/docs/knowledgebase/add-column#adding-a-new-column" class="hash-link" aria-label="Direct link to Adding a new column" title="Direct link to Adding a new column">​</a></h2>
<p>Now let's say we're going to add a new column called <code>favoriteNumber</code>, which will be a <code>Float64</code>.
We can do this using the <a href="https://clickhouse.com/docs/sql-reference/statements/alter/column#add-column"><code>ALTER TABLE...ADD COLUMN</code></a> clause:</p>
<div class="wrapper_EBtA" style="height:37.7px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">ALTER TABLE events 
ADD COLUMN favoriteNumber Float64 DEFAULT 7;
</code></pre></div><div class="activity_PoTP"></div></div>
<p>If we query the <code>events</code> table, we'll see the following output:</p>
<div class="wrapper_EBtA" style="height:56.550000000000004px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">┌───────date─┬─name───┬─favoriteNumber─┐
│ 2024-12-18 │ Alexey │              7 │
└────────────┴────────┴────────────────┘
</code></pre></div><div class="activity_PoTP"></div></div>
<p>The <code>Alexey</code> row defaults to 7 since that column didn't exist when we added that row.
Next, let's add another column:</p>
<div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">INSERT INTO events (name) VALUES ('Tyler');
</code></pre></div><div class="activity_PoTP"></div></div>
<p>If we query the <code>events</code> table, we'll see the following output:</p>
<div class="wrapper_EBtA" style="height:131.95000000000002px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">┏━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃       date ┃ name   ┃ favoriteNumber ┃
┡━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ 2024-12-18 │ Tyler  │              7 │
├────────────┼────────┼────────────────┤
│ 2024-12-18 │ Alexey │              7 │
└────────────┴────────┴────────────────┘
</code></pre></div><div class="activity_PoTP"></div></div>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="modifying-a-columns-default-value">Modifying a column's default value<a href="https://clickhouse.com/docs/knowledgebase/add-column#modifying-a-columns-default-value" class="hash-link" aria-label="Direct link to Modifying a column's default value" title="Direct link to Modifying a column's default value">​</a></h2>
<p>If we modify the <code>favoriteNumber</code> column to have a different type using the <a href="https://clickhouse.com/docs/sql-reference/statements/alter/column#modify-column"><code>ALTER TABLE...MODIFY COLUMN</code></a> clause, things get interesting:</p>
<div class="wrapper_EBtA" style="height:37.7px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">ALTER TABLE events 
MODIFY COLUMN favoriteNumber Float64 DEFAULT 99;
</code></pre></div><div class="activity_PoTP"></div></div>
<p>If we query <code>events</code> again, we'll see this output:</p>
<div class="wrapper_EBtA" style="height:131.95000000000002px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">┏━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃       date ┃ name   ┃ favoriteNumber ┃
┡━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ 2024-12-18 │ Tyler  │              7 │
├────────────┼────────┼────────────────┤
│ 2024-12-18 │ Alexey │             99 │
└────────────┴────────┴────────────────┘
</code></pre></div><div class="activity_PoTP"></div></div>
<p><code>Tyler</code> keeps a value of <code>7</code>, which was the default when that row was created.
<code>Alexey</code> picks up the new default of <code>99</code> because the <code>favoriteNumber</code> column didn't exist when that row was created.</p>
<p>If we want the <code>Alexey</code> row to use the current default right away, we need to call <a href="https://clickhouse.com/docs/sql-reference/statements/optimize"><code>OPTIMIZE TABLE</code> </a> to force current defaults to be written to disk:</p>
<div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">OPTIMIZE TABLE events;
</code></pre></div><div class="activity_PoTP"></div></div>
<p>Once we've done that, let's say we change the default value again:</p>
<div class="wrapper_EBtA" style="height:37.7px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">ALTER TABLE events 
MODIFY COLUMN favoriteNumber Float64 DEFAULT 21;
</code></pre></div><div class="activity_PoTP"></div></div>
<p>And then insert another row:</p>
<div class="wrapper_EBtA" style="height:18.85px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">INSERT INTO events (name) VALUES ('Tanya');
</code></pre></div><div class="activity_PoTP"></div></div>
<p>Finally, let's query <code>events</code> one more time:</p>
<div class="wrapper_EBtA" style="height:169.65px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">┏━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃       date ┃ name   ┃ favoriteNumber ┃
┡━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ 2024-12-18 │ Alexey │             99 │
├────────────┼────────┼────────────────┤
│ 2024-12-18 │ Tyler  │              7 │
├────────────┼────────┼────────────────┤
│ 2024-12-18 │ Tanya  │             21 │
└────────────┴────────┴────────────────┘
</code></pre></div><div class="activity_PoTP"></div></div>
<p><code>Tanya</code> picks up the new default of <code>21</code>, but <code>Alexey</code> has the old default of <code>99</code>.</p>
<h2 class="anchor anchorWithStickyNavbar_LWe7" id="controlling-column-position-in-table">Controlling column position in table<a href="https://clickhouse.com/docs/knowledgebase/add-column#controlling-column-position-in-table" class="hash-link" aria-label="Direct link to Controlling column position in table" title="Direct link to Controlling column position in table">​</a></h2>
<p>When we add a new column, by default it will be added at the end of the table.
But, we can use the <code>FIRST</code> and <code>AFTER</code> clauses to control where a column is positioned.</p>
<p>For example, if we wanted to add a column called <code>favoriteColor</code> after the <code>name</code> column, we could do this:</p>
<div class="wrapper_EBtA" style="height:37.7px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">ALTER TABLE events
ADD COLUMN favoriteColor String DEFAULT 'Yellow' AFTER name;
</code></pre></div><div class="activity_PoTP"></div></div>
<p>Let's query <code>events</code>:</p>
<div class="wrapper_EBtA" style="height:169.65px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">┏━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃       date ┃ name   ┃ favoriteColor ┃ favoriteNumber ┃
┡━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ 2024-12-18 │ Alexey │ Yellow        │             99 │
├────────────┼────────┼───────────────┼────────────────┤
│ 2024-12-18 │ Tyler  │ Yellow        │              7 │
├────────────┼────────┼───────────────┼────────────────┤
│ 2024-12-18 │ Tanya  │ Yellow        │             21 │
└────────────┴────────┴───────────────┴────────────────┘
</code></pre></div><div class="activity_PoTP"></div></div>
<p>And if we wanted to add a column <code>favoriteDatabase</code> and have that be first in the list, we could do this:</p>
<div class="wrapper_EBtA" style="height:37.7px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">ALTER TABLE events
ADD COLUMN favoriteDatabase String DEFAULT 'ClickHouse' FIRST;
</code></pre></div><div class="activity_PoTP"></div></div>
<div class="wrapper_EBtA" style="height:169.65px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-text"><code class="language-text">┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ favoriteDatabase ┃       date ┃ name   ┃ favoriteColor ┃ favoriteNumber ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ ClickHouse       │ 2024-12-18 │ Tanya  │ Yellow        │             21 │
├──────────────────┼────────────┼────────┼───────────────┼────────────────┤
│ ClickHouse       │ 2024-12-18 │ Alexey │ Yellow        │             99 │
├──────────────────┼────────────┼────────┼───────────────┼────────────────┤
│ ClickHouse       │ 2024-12-18 │ Tyler  │ Yellow        │              7 │
└──────────────────┴────────────┴────────┴───────────────┴────────────────┘
</code></pre></div><div class="activity_PoTP"></div></div>
<p>And let's have a look at the table definition:</p>
<div class="wrapper_EBtA" style="height:37.7px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">SHOW CREATE TABLE events
FORMAT LineAsString
</code></pre></div><div class="activity_PoTP"></div></div>
<div class="wrapper_EBtA" style="height:188.5px"><div style="position:absolute;left:-9999px;top:-9999px;opacity:0;pointer-events:none;width:1px;height:1px;overflow:hidden"><pre class="language-sql"><code class="language-sql">CREATE TABLE default.`clickhouse-local-ab404c86-56cc-495b-ad1d-fb343cac3bc0events`
(
    `favoriteDatabase` String DEFAULT 'ClickHouse',
    `date` Date DEFAULT today(),
    `name` String,
    `favoriteColor` String DEFAULT 'Yellow',
    `favoriteNumber` Float64 DEFAULT 21
)
ENGINE = MergeTree
ORDER BY date
</code></pre></div><div class="activity_PoTP"></div></div><div class="container_iJTo margin-vert--md"><time datetime="2024-12-18T00:00:00.000Z">Dec 18, 2024</time> · <!-- -->4 min read</div>]]></content:encoded>
            <category>Data Modelling</category>
        </item>
    </channel>
</rss>